Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2005 Forums
 Transact-SQL (2005)
 An alternative to while loops to improve SP perf

Author  Topic 

anilofar
Starting Member

9 Posts

Posted - 2010-11-30 : 01:43:54
I have the following two tables

Table1
Id State Date
1 A 11/20/2010
1 A 11/21/2010
1 B 11/23/2010
2 A 11/20/2010
2 B 11/21/2010
2 B 11/22/2010
3 B 11/20/2010
3 A 11/23/2010
3 C 11/24/2010

Table2
Id Date
1 11/20/2010
2 11/21/2010
3 11/22/2010
4 11/23/2010
5 11/24/2010

And, this is the information that I want from these tables. This is basically a count of all ids from Table1 in each state. If an entry does not exist for a specific date in Table1 then the count for that date should be from last date.

ResultTable
Date State Count
11/20/2010 A 2
11/20/2010 B 1
11/21/2010 A 1
11/21/2010 B 2
11/22/2010 A 1
11/22/2010 B 2
11/23/2010 A 1
11/23/2010 B 2
11/24/2010 C 1
11/24/2010 B 2

This is what I am currently doing

SET @NumberRecords = @@ROWCOUNT
SET @RowCount = 1

WHILE @RowCount <= @NumberRecords
BEGIN
SELECT @tempDate = Date from Table2 WHERE Id = @RowCount

INSERT INTO #ResultTable
SELECT @tempDate, State, Count(Id)
FROM
(
SELECT A.Id
, (SELECT TOP 1 State
FROM Table1 B
WHERE B.Date <= @tempDate
AND A.Id = B.Id
ORDER BY DATE Desc) AS State
FROM Table1 A
)[X]
GROUP BY State

SET @RowCount = @RowCount + 1
END

I see a huge perf issue since I am dealing with a lot of data here.

I believe there should be another way to do this. Any help would be greatly appreciated.

Thanks

dataguru1971
Master Smack Fu Yak Hacker

1464 Posts

Posted - 2010-11-30 : 06:59:49
[code]
Create Table TableA (ID int not null, [State] char(1) not null, [Date] datetime not null)
go

Insert into TableA
Select 1, 'A', '11/20/2010' UNION
Select 1, 'A', '11/21/2010' UNION
Select 1, 'B', '11/23/2010' UNION
Select 2, 'A', '11/20/2010' UNION
Select 2, 'B', '11/21/2010' UNION
Select 2, 'B', '11/22/2010' UNION
Select 3, 'B', '11/20/2010' UNION
Select 3, 'A', '11/23/2010' UNION
Select 3, 'C', '11/24/2010'

Create Table TableB (ID int not null, [Date] datetime not null)
go
Insert into TableB
Select 1, '11/20/2010' UNION
Select 2, '11/21/2010' UNION
Select 3, '11/22/2010' UNION
Select 4, '11/23/2010' UNION
Select 5, '11/24/2010'

go

Select COUNT(*) as NumRecords,
TableB.[Date],
TableA.[State]
FROM TableB inner join TableA ON
TableB.ID = TableB.ID
Where TableA.Date <= TableB.Date
Group By TableB.[Date],TableA.State
Order by TableB.[Date] desc, [State] asc


drop table TableA
Drop table TableB
[/code]

Not sure what you are doing, but this might work for you or get you out of the looping mindset.



Poor planning on your part does not constitute an emergency on my part.

Go to Top of Page

anilofar
Starting Member

9 Posts

Posted - 2010-11-30 : 14:02:11
Thanks for the response. But I can't do a join on these tables based on id. Since the id for these two tables mean two seperate things. Table2 id is only used to iterate through while loop.

I also can't do a join on these two tables by date and group because there is some info missing from Table1 for some dates.

The result table is essentially a trend line. I want to be able to calculate the current date's count while also taking into account the count for previous dates. I hope I have made this clear.


Go to Top of Page

Lamprey
Master Smack Fu Yak Hacker

4614 Posts

Posted - 2010-11-30 : 19:54:11
I don't understand your requirements or your sample output doesn't match the sample data or both.

- Why does "11/21/2010 B 2" have a count of 2? "B" only appears once for that date. Same with most of the B's.
- Why does C only appear once? Is that because he first instance of C appears on 11/24/2010? Why doesn't C have a zero count for the other previous days?
Go to Top of Page

anilofar
Starting Member

9 Posts

Posted - 2010-11-30 : 21:28:26
quote:
- Why does "11/21/2010 B 2" have a count of 2? "B" only
appears once for that date. Same with most of the B's.


11/21/2010 B has count of 2 because 2 has state B at 11/21/2010 and 3 has state B at 11/20/2010. I need to find accumulating state count where state info is missing for a some id, and dates

quote:
- Why does C only appear once? Is that because he first instance of C appears on 11/24/2010? Why doesn't C have a zero count for the other previous days?


C only appears once here, because the first occurence of State C is at 11/24/2010. This is the only data I have to work with, there is no 0 count for previous dates. Also, there could be missing info for some ids at some specific dates, and I need to treat this as if the state info is carried forward from previous dates for that id.


I tried to add this additional info to Table1 before calculating count. However, I do not see any perf difference. I am working with 300000+ rows of data. Below is the code that does this (maybe this can be optimized?)


DECLARE @Table1 TABLE
(Id INT, [State] CHAR, [TDate] DATETIME)

INSERT @Table1
SELECT 1, 'A', '11/20/2010' UNION
SELECT 1, 'A', '11/21/2010' UNION
SELECT 1, 'B', '11/23/2010' UNION
SELECT 2, 'A', '11/20/2010' UNION
SELECT 2, 'B', '11/21/2010' UNION
SELECT 2, 'B', '11/22/2010' UNION
SELECT 3, 'B', '11/20/2010' UNION
SELECT 3, 'A', '11/23/2010' UNION
SELECT 3, 'C', '11/24/2010'

DECLARE @Table2 TABLE
([DDate] DATETIME)
INSERT @Table2
SELECT '11/20/2010' UNION
SELECT '11/21/2010' UNION
SELECT '11/22/2010' UNION
SELECT '11/23/2010' UNION
SELECT '11/24/2010'

INSERT @Table1
SELECT Id, [State], DDate
FROM
(
SELECT DDate, ID, TDate, State,
ROW_NUMBER() OVER (PARTITION BY Id, dDate Order by Id, Diff, dDate) as row
FROM
(
SELECT DDate, t.Id, TDate, t.State, CAST(DATEDIFF(day, TDate, dDate) as int) as Diff
FROM @Table2 d, @Table1 t
WHERE CAST(DATEDIFF(DAY, TDate, DDate) as int) >= 1
AND DDate NOT IN (SELECT TDate from @Table1 WHERE Id = t.Id)
)[x]
)[y]
WHERE row = 1

SELECT TDate, State, COUNT(Id)
FROM @Table1
GROUP BY TDate, State
ORDER BY TDate, State

Go to Top of Page

Lamprey
Master Smack Fu Yak Hacker

4614 Posts

Posted - 2010-12-01 : 12:03:19
quote:
Originally posted by anilofar

11/21/2010 B has count of 2 because 2 has state B at 11/21/2010 and 3 has state B at 11/20/2010. I need to find accumulating state count where state info is missing for a some id, and dates
I still do not understand. What is the second 2 in your statement? Is that the ID? If so, how does that play into calculating the count?

Any chance you could take another shot at articulating the requirements in more detail?


I thought the code snippet you provided was producing the results you wanted and you just wanted to improve the performance. I took your code and modified it a little so I could get it to run and it produces results that are way different from what you want as output. If you can modify the code to produce the results you want in addition to articulating your requirements, I'm sure we can help you out. But, if you are struggeling with getting the correct output then we (at least I) need more detail information about how to get said output. (agian back to requirements :) )
DECLARE @NumberRecords INT
DECLARE @RowCount INT
DECLARE @tempDate DATETIME

SET @NumberRecords = (SELECT COUNT(*) FROM TableB)
SET @RowCount = 1

IF OBJECT_ID('TempDB..#ResultTable','U') IS NOT NULL
DROP TABLE #ResultTable

CREATE TABLE #ResultTable ([Date] DATETIME, State CHAR(1), ID INT)

WHILE @RowCount <= @NumberRecords
BEGIN
SELECT @tempDate = [Date] from TableB WHERE Id = @RowCount

INSERT #ResultTable
SELECT @tempDate AS [Date], State, Count(Id) ID
FROM
(
SELECT A.Id
, (SELECT TOP 1 State
FROM TableA AS B
WHERE B.[Date] <= @tempDate
AND A.Id = B.Id
ORDER BY DATE Desc) AS State
FROM TableA AS A
)[X]
GROUP BY State

SET @RowCount = @RowCount + 1
END


Produces:
Date                    State ID
----------------------- ----- -----------
2010-11-20 00:00:00.000 A 6
2010-11-20 00:00:00.000 B 3
2010-11-21 00:00:00.000 A 3
2010-11-21 00:00:00.000 B 6
2010-11-22 00:00:00.000 A 3
2010-11-22 00:00:00.000 B 6
2010-11-23 00:00:00.000 A 3
2010-11-23 00:00:00.000 B 6
2010-11-24 00:00:00.000 B 6
2010-11-24 00:00:00.000 C 3
Go to Top of Page

anilofar
Starting Member

9 Posts

Posted - 2010-12-01 : 13:44:25
Ok. Lets try this again.

quote:
I still do not understand. What is the second 2 in your statement? Is that the ID? If so, how does that play into calculating the count?



The second 2 is the id.

Sorry for all this confusion. I realized I am missing another table.

Table1
Id State Date
1 A 11/20/2010
1 A 11/21/2010
1 B 11/23/2010
2 A 11/20/2010
2 B 11/21/2010
2 B 11/22/2010
3 B 11/20/2010
3 A 11/23/2010
3 C 11/24/2010

Table2
Id Date
1 11/20/2010
2 11/21/2010
3 11/22/2010
4 11/23/2010
5 11/24/2010

Table3
Id
1
2
3


SET @NumberRecords = @@ROWCOUNT
SET @RowCount = 1

WHILE @RowCount <= @NumberRecords
BEGIN
SELECT @tempDate = Date from Table2 WHERE Id = @RowCount

INSERT INTO #ResultTable
SELECT @tempDate, State, Count(Id)
FROM
(
SELECT A.Id
, (SELECT TOP 1 State
FROM Table1 B
WHERE B.Date <= @tempDate
AND A.Id = B.Id
ORDER BY DATE Desc) AS State
FROM Table3 A
)[X]
GROUP BY State

SET @RowCount = @RowCount + 1
END


My requirements here are that I have Table3 which contains all the Tasks. Table1 which is a list of all states that a task have taken. Table2 the date range for which I need to find state count information. The thing is that I do not have state info for each task for all dates. However, if a state info is missing for one day, it should carry over from the previous day.

And,I am trying to generate the number of task counts for each state for all dates in range.
Go to Top of Page

Lamprey
Master Smack Fu Yak Hacker

4614 Posts

Posted - 2010-12-01 : 16:29:14
I'm not sure if this covers all the scenarios, but it seems to work for your sample data:
SELECT 
D.[Date],
D.State,
COUNT(D.ID) AS MyCount
FROM
(
SELECT
*
FROM
(
SELECT
*
FROM
-- Assumeing these are already distinct
(SELECT ID FROM TableC) AS C
CROSS JOIN
(SELECT [Date] FROM TableB) AS B
) AS Combo
CROSS APPLY
(
SELECT TOP 1 State
FROM TableA AS A
WHERE A.[Date] <= Combo.[Date]
AND A.ID = Combo.ID
ORDER BY [Date] DESC

) AS T
) AS D
GROUP BY
D.[Date],
D.State
ORDER BY
D.[Date],
D.State
Go to Top of Page

anilofar
Starting Member

9 Posts

Posted - 2010-12-01 : 19:23:16
Thanks Lamprey! That works. Though I only see a very slight improvement in perf. I think, I will increase the date intervals to speed up query.

Go to Top of Page

Lamprey
Master Smack Fu Yak Hacker

4614 Posts

Posted - 2010-12-02 : 11:56:59
Yeah, it's still doing REBAR (in essence a loop). So, I wouldn't expect too much of an improvement. If I have any time in the next day or two, I might see if I can do it in a more set-based manner.
Go to Top of Page
   

- Advertisement -