Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
Author |
Topic |
deonvisser
Starting Member
16 Posts |
Posted - 2014-05-12 : 22:45:21
|
HiI am stepping through table row by row and processing data. I am not using cursors for this process. Everything seems to be working fine and then I run out of memory.What I am trying to do:A column in a table contains values 013|014|015|016 etc. Each of these values separated by | should be in its own row and not in on row as I depicted. So I wrote some code to pull each number out row by row and write them as separate rows back to the table. The logic works 100% except that it bombs out with a memory error. I seems that memory is not being released after a statement is executed.This is an example of the dataVoIS Local All Of RSA Virtual Card Acquiring no 0.28 Monday Friday 00:00 06:59 0101250|0121250|0211250|0311250|0411250|0431250|0511250The procedure will work fine on a few lines but will bomb on thirty line.This is the code that I run.....-- THIS IS THE CONVERSION PROCESS TO REMOVE ALL THE | FROM THE DATA AN CREATE SEPERATE ROWS FOR EACH DIAL PREFIX WHILE EXISTS (SELECT * FROM tbl_MitsolRate) BEGIN-- CHECK IF THE RATE TABLE ROW CONTAINS | SELECT TOP(1)[Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List] FROM tbl_MitsolRate IF @@ROWCOUNT > 0 BEGIN INSERT INTO tbl_Temp_MitsolRate ([Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List]) SELECT TOP(1) [Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List] FROM tbl_MitsolRate DELETE TOP(1) FROM tbl_MitsolRate -- CHECK IF THE TEMP TABLE ROW CONTAINS | WHILE EXISTS (SELECT * FROM tbl_Temp_MitsolRate) BEGIN SELECT [Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], RTRIM(SubString([Destination Prefix List], 1, CharIndex('|', [Destination Prefix List])-1)) FROM qry_Mitsol_Temp IF @@ROWCOUNT > 0 BEGIN INSERT INTO tbl_New_MitsolRate ([Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List]) SELECT [Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], RTRIM(SubString([Destination Prefix List], 1, CharIndex('|', [Destination Prefix List])-1)) FROM qry_Mitsol_Temp UPDATE tbl_Temp_MitsolRate SET [Destination Prefix List] = RTRIM(SubString([Destination Prefix List], CharIndex('|', [Destination Prefix List])+ 1, LEN([Destination Prefix List]))) END ELSE BEGIN INSERT INTO tbl_New_MitsolRate ([Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List]) SELECT [Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List] FROM qry_Mitsol_Temp1 DELETE FROM tbl_Temp_MitsolRate END END ENDPlease can someone help. There may be a better way to code this? |
|
nagino
Yak Posting Veteran
75 Posts |
Posted - 2014-05-13 : 01:23:50
|
[code]WITH WORK ( [Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List], flg) AS (SELECT [Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List], CHARINDEX('|', [Destination Prefix List])FROM @tbl_MitsolRateUNION ALLSELECT [Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], LEFT([Destination Prefix List], CHARINDEX('|', [Destination Prefix List]) - 1), 0FROM Work WHERE 0 < flgUNION ALLSELECT [Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], SUBSTRING([Destination Prefix List], CHARINDEX('|', [Destination Prefix List]) + 1, LEN([Destination Prefix List])), CHARINDEX('|', SUBSTRING([Destination Prefix List], CHARINDEX('|', [Destination Prefix List]) + 1, LEN([Destination Prefix List])))FROM Work WHERE 0 < flg)INSERT INTO tbl_New_MitsolRateSELECT [Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List]FROM WORK WHERE flg = 0[/code]-------------------------------------From JapanSorry, my English ability is limited. |
|
|
deonvisser
Starting Member
16 Posts |
Posted - 2014-05-13 : 06:22:56
|
Thank you for your reply nagino.Unfortunately I cannot get it to work. Lots of errors. |
|
|
stepson
Aged Yak Warrior
545 Posts |
Posted - 2014-05-13 : 06:31:03
|
one way to split :declare @vcString as varchar(500)declare @xmlVar as xmlSET @vcString ='0101250|0121250|0211250|0311250|0411250|0431250|0511250'SET @vcString = replace(@vcString,'|','</Row><Row>')--print @vcStringSET @vcString='<Rows><Row>' + @vcString + '</Row></Rows>'SET @xmlVar=CAST(@vcString as XML)--print @vcStringSELECT t.u.value('.','varchar(50)')FROM @xmlVar.nodes('Rows/Row')t(u) sabinWeb MCP |
|
|
Ifor
Aged Yak Warrior
700 Posts |
Posted - 2014-05-13 : 07:25:25
|
It is normal to split delimited strings with a number/tally table (google this)As you have not provided data, or even said what the primary key is, this is difficult to show and test.Something like the following should work if you replace <tbl_MitsolRatePK> with the actual primary key.With no test data, the details are up to you.WITH N1(N) AS (SELECT N FROM (VALUES(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)) d(N)),N2(N) AS (SELECT 1 FROM N1 a, N1 b),N4(N) AS (SELECT 1 FROM N2 a, N2 b) ,N(N) AS (SELECT ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) FROM N4)--select * from N,DestSplit1AS( SELECT <tbl_MitsolRatePK> ,SUBSTRING('|' + [Destination Prefix List], N+1, 255) AS Dest FROM tbl_MitsolRate CROSS JOIN N WHERE SUBSTRING('|' + [Destination Prefix List], N.N, 1) = '|'),DestSplitAS( SELECT <tbl_MitsolRatePK> ,CASE WHEN CHARINDEX('|', Dest) > 0 THEN LEFT(Dest, CHARINDEX('|', Dest) -1) ELSE Dest END AS Dest FROM DestSplit1)INSERT INTO tbl_New_MitsolRate ([Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List])SELECT S.[Service Type], S.Origin, S.Destination, S.[Rate Per Minute], S.[Minimum Charge], S.[Time Based Rate], S.[Time Based From Day],S.[Time Based To Day], S.[Time Based From Time], S.[Time Based To Time], D.DestFROM tbl_MitsolRate S JOIN DestSplit D ON S.<tbl_MitsolRatePK> = D.<tbl_MitsolRatePK>; |
|
|
deonvisser
Starting Member
16 Posts |
Posted - 2014-05-13 : 16:12:12
|
Hi AllThanks for your replies. Ifor, this is an example of the data...SERVICE ORIGIN DESTINATION DIAL PREFIXVoIS Local All Of RSA Virtual Card Acquiring 0101250|0121250|0211250VoIS National All Of RSA Virtual Card Acquiring 0103250|01271250|02150|03113VoIS Int All Of RSA Virtual Card Acquiring 01016650|0121250As you can see the table does not contain a primary keyThe result I need is thisSERVICE ORIGIN DESTINATION DIAL PREFIXVoIS Local All Of RSA Virtual Card Acquiring 0101250VoIS Local All Of RSA Virtual Card Acquiring 0121250VoIS Local All Of RSA Virtual Card Acquiring 0211250VoIS National All Of RSA Virtual Card Acquiring 0103250VoIS National All Of RSA Virtual Card Acquiring 01271250VoIS National All Of RSA Virtual Card Acquiring 02150VoIS National All Of RSA Virtual Card Acquiring 03113VoIS Int All Of RSA Virtual Card Acquiring 01016650VoIS Int All Of RSA Virtual Card Acquiring 0121250I hope this makes sense?Let me know if you need more infoThank you for your assistance |
|
|
deonvisser
Starting Member
16 Posts |
Posted - 2014-05-13 : 16:17:51
|
Hmm, the Forum does not maintain my formatting. BasicallySERVICE ORIGIN DESTINATION DIAL PREFIX are the Col NamesAs you can see, only the last col that contains the [DIAL PREFIX] has multiple Prefixes separated by |Each of these prefixes must be split into new rows reatining the data of the other columns (SERVICE ORIGIN DESTINATION) |
|
|
TG
Master Smack Fu Yak Hacker
6065 Posts |
Posted - 2014-05-13 : 16:46:08
|
quote: Hmm, the Forum does not maintain my formatting
use code tags around formatted code to maintain the formatting:[code]formatted code[/code]read the faq for more options. Be One with the OptimizerTG |
|
|
TG
Master Smack Fu Yak Hacker
6065 Posts |
Posted - 2014-05-13 : 17:40:17
|
quote: Originally posted by deonvisser BasicallySERVICE ORIGIN DESTINATION DIAL PREFIX are the Col NamesAs you can see, only the last col that contains the [DIAL PREFIX] has multiple Prefixes separated by |Each of these prefixes must be split into new rows reatining the data of the other columns (SERVICE ORIGIN DESTINATION)
This looks very simple and requires no looping at all. The final query would look something like this:select yt.SERVICE , yt.ORIGIN , yt.DESTINATION , ca.outputCol as [prefix]from yourTable as ytcross apply dbo.fn_split([DIAL PREFIX], '|') ca dbo.fn_split() could be any "split function" Search for split functions or parse CSV for many options.EDIT:this is an article by Jeff Moden who thoroughly researches whatever he's writing about so I have no doubt that what he suggests will work fast and well. Although I have to admit I haven't done more than glance through this article yet.Be One with the OptimizerTG |
|
|
deonvisser
Starting Member
16 Posts |
Posted - 2014-05-14 : 02:37:52
|
Thank you for all your replies.TG, this is what I have doneselect [Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List], ca.outputCol as [prefix]from tbl_MitsolRate as ytcross apply dbo.split([Destination Prefix List], '|') caCREATE FUNCTION [dbo].[SPLIT] ( @DELIMITER VARCHAR(5), @LIST VARCHAR(MAX) ) RETURNS @TABLEOFVALUES TABLE ( ROWID SMALLINT IDENTITY(1,1), [VALUE] VARCHAR(MAX) ) AS BEGIN DECLARE @LENSTRING INT WHILE LEN( @LIST ) > 0 BEGIN SELECT @LENSTRING = (CASE CHARINDEX( @DELIMITER, @LIST ) WHEN 0 THEN LEN( @LIST ) ELSE ( CHARINDEX( @DELIMITER, @LIST ) -1 ) END ) INSERT INTO @TABLEOFVALUES SELECT SUBSTRING( @LIST, 1, @LENSTRING ) SELECT @LIST = (CASE ( LEN( @LIST ) - @LENSTRING ) WHEN 0 THEN '' ELSE RIGHT( @LIST, LEN( @LIST ) - @LENSTRING - 1 ) END ) END RETURN END However it does not like the output col it fails. |
|
|
deonvisser
Starting Member
16 Posts |
Posted - 2014-05-14 : 04:21:35
|
Hi GuysAnyone out there that can help. I am desperate to get this working and will even pay to get it working.Thanks |
|
|
deonvisser
Starting Member
16 Posts |
Posted - 2014-05-14 : 05:46:29
|
Not to worry, I found something that did the job perfectly....create FUNCTION [dbo].[Split](@String varchar(MAX), @Delimiter char(1)) returns @temptable TABLE (items varchar(MAX)) as begin declare @idx int declare @slice varchar(8000) select @idx = 1 if len(@String)<1 or @String is null return while @idx!= 0 begin set @idx = charindex(@Delimiter,@String) if @idx!=0 set @slice = left(@String,@idx - 1) else set @slice = @String if(len(@slice)>0) insert into @temptable(Items) values(@slice) set @String = right(@String,len(@String) - @idx) if len(@String) = 0 break end return end;INSERT INTO tbl_mitsolrate_new ([Service Type], Origin, Destination, [Rate Per Minute], [Minimum Charge], [Time Based Rate], [Time Based From Day], [Time Based To Day], [Time Based From Time], [Time Based To Time], [Destination Prefix List])select t1.[service type], t1.Origin, t1.Destination, t1.[Rate Per Minute], t1.[Minimum Charge], t1.[Time Based Rate], t1.[Time Based From Day], t1.[Time Based To Day], t1.[Time Based From Time], t1.[Time Based To Time], s.items [Destination Prefix List]from tbl_mitsolrate t1outer apply dbo.split(t1.[Destination Prefix List], '|') s Thank you for all your input. Much appreciated. |
|
|
|
|
|
|
|