Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2000 Forums
 SQL Server Development (2000)
 Index for Recordset Paging

Author  Topic 

SamC
White Water Yakist

3467 Posts

Posted - 2004-01-07 : 09:22:25
I'm implementing recordset paging in a stored procedure. There are many sort options and filter options. The usual technique of placing the sorted material in a temporary table like this:

CREATE TABLE #MyOrder (
MyID INT IDENTITY (1,1) PRIMARY KEY,
UserID INT )

After the table is populated, then records N to N+M are extracted for the requested page, and UserID is used to find the needed records. Several joins exist to retrieve the data, and I'm wondering how best to index UserID in the temp table?

CREATE UNIQUE CLUSTERED INDEX abc ON #MyOrder (MyID, UserID) -- I'd get rid of the primary key above for this

CREATE UNIQUE INDEX abc ON #MyOrder (UserID)
-------------------------------------------------
And a separate question:

Here's an observation I made that I wonder if anyone can confirm for me. I once ran some tests populating a similar table and found that it ran much faster if I populated the table first, then created the index before using the data. It makes some sense if you think about it. Population is atomic (single insert), the indexing of the data could be faster if it was done in a single swoop, after the table was filled.

Sam

X002548
Not Just a Number

15586 Posts

Posted - 2004-01-07 : 10:13:41
Sam,

Why a temp table?

It'll disappear after the sproc ends...

So why put more data in than is required...seems like you'll only be able to present 1 page at a time anyway....

And yes bilding the indexes after the fact sometimes make sense...

But can you give use more background on what your doing?



Brett

8-)
Go to Top of Page

SamC
White Water Yakist

3467 Posts

Posted - 2004-01-07 : 10:56:32
Hi Brett,

The direction I've taken to retrieve paged data for web display is derived from articles on 4-Guys, Mr. Cross Join's BLOG and posts here on SQLTEAM.

There are several methods to page resultset data, but the consensus on optimal recordset paging (in SQL 2000) tasks the stored procedure to build a query for the result set, then inserting all the data (or a unique identifier) into a temporary table with a sequence column built using IDENTITY(1,1). The sequenced record (column) numbers are then used to easily find the *page* which is a starting row number, and a number of rows following the starting point.

In my case, the query has multiple filters, and 3 ordering levels which can be either ascending or descending. This solution is pretty good for a code-it-yourself solution, but paging will be a big part of the web future so I like to think Yukon will have better support for paging. (anyone have info on this?) Paging data ought to be easier than what we do in T-SQL.

The temporary table is a good choice if there is a possability of it being used by more than 1 user at a time. I might convert the temporary table to a local table variable - once everything is debuged.

I've included the code to show how complex the whole paging process becomes. I think the idea for ascending / descending ORDER BY came from Jeff.

I may draw some flack about the insertion of HTML in the resulting data, but hey, this is preproduction testing, not commercial stuff yet.

Sam

/* For paging, we retrieve the selected UserIDs, in order, into a temporary table  */
CREATE TABLE #TempRowLookup (
RowID INT IDENTITY(0,1) PRIMARY KEY, -- We can retrieve any sequence of ordered rows for a specific page using RowID
UserID INT
)

/* We don't need records beyond the end of page */
DECLARE @RowLimit INT
SET @RowLimit = @PageSize * (@CurrentPage + 1)
SET ROWCOUNT @RowLimit

INSERT INTO #TempRowLookup (UserID)
SELECT U.UserID
FROM dbo.Users U
LEFT OUTER JOIN dbo.CourseRecords CR
ON CR.UserID = U.UserID
AND CR.CourseID = @CourseID
AND CR.CRecordFinish BETWEEN @CompleteStart AND @CompleteFinish
LEFT OUTER JOIN dbo.Courses CS ON CS.CourseID = CR.CourseID
LEFT OUTER JOIN dbo.Branches B on U.BranchID=B.BranchID
LEFT OUTER JOIN dbo.Centers C on U.CenterID=C.CenterID

WHERE U.ClientID = @ClientID
AND (@Email IS NULL OR Email LIKE @Email)
AND (@Firstname IS NULL OR Firstname LIKE @Firstname)
AND (@Middlename IS NULL OR Middlename LIKE @Middlename)
AND (@Lastname IS NULL OR Lastname LIKE @Lastname)
AND (@Username IS NULL OR Username LIKE @Username)
AND (@EmployeeType IS NULL OR EmployeeType LIKE @EmployeeType)
AND (@CenterID IS NULL OR U.CenterID = @CenterID OR U.CenterID = 0)
AND (@BranchID IS NULL OR U.BranchID = @BranchID)
AND (@Wildcard IS NULL OR B.Branchname LIKE @Branchname)
AND (@Offline IS NULL OR U.Offline = @Offline)
AND (@InactiveMask IS NULL OR (U.Inactive & @InactiveMask <> 0))
AND (@Completed IS NULL
OR (@Completed=0 AND CRecordFinish IS NULL)
OR (@Completed=1 AND CRecordFinish BETWEEN @CompleteStart AND @CompleteFinish)
)
AND (@Acknowledge IS NULL
OR (@Acknowledge=1 AND CRAcknowledge=1)
OR (@Acknowledge=0 AND (CRAcknowledge = 0 or CRAcknowledge = -1)))

ORDER BY
CASE WHEN @Order1 = 'Username' THEN Username
WHEN @Order1 = 'Firstname' THEN Firstname
WHEN @Order1 = 'Middlename' THEN Middlename
WHEN @Order1 = 'Lastname' THEN Lastname
WHEN @Order1 = 'EmployeeType' THEN EmployeeType
WHEN @Order1 = 'Centername' THEN Centername
WHEN @Order1 = 'Branchname' THEN Branchname
WHEN @Order1 = 'Email' THEN Email
WHEN @Order1 = 'CompletedDate' THEN CONVERT(VARCHAR, CRecordFinish, 120)
ELSE 'nosorta'
END ASC ,
CASE WHEN @Order1 = 'Username-DESC' THEN Username
WHEN @Order1 = 'Firstname-DESC' THEN Firstname
WHEN @Order1 = 'Middlename-DESC' THEN Middlename
WHEN @Order1 = 'Lastname-DESC' THEN Lastname
WHEN @Order1 = 'EmployeeType-DESC' THEN EmployeeType
WHEN @Order1 = 'Centername-DESC' THEN Centername
WHEN @Order1 = 'Branchname-DESC' THEN Branchname
WHEN @Order1 = 'Email-DESC' THEN Email
WHEN @Order1 = 'CompletedDate-DESC' THEN CONVERT(VARCHAR, CRecordFinish, 120)
ELSE 'nosortb'
END DESC ,
CASE WHEN @Order2 = 'Username' THEN Username
WHEN @Order2 = 'Firstname' THEN Firstname
WHEN @Order2 = 'Middlename' THEN Middlename
WHEN @Order2 = 'Lastname' THEN Lastname
WHEN @Order2 = 'EmployeeType' THEN EmployeeType
WHEN @Order2 = 'Centername' THEN Centername
WHEN @Order2 = 'Branchname' THEN Branchname
WHEN @Order2 = 'Email' THEN Email
WHEN @Order1 = 'CompletedDate' THEN CONVERT(VARCHAR, CRecordFinish, 120)
ELSE 'nosortc'
END ASC ,
CASE WHEN @Order2 = 'Username-DESC' THEN Username
WHEN @Order2 = 'Firstname-DESC' THEN Firstname
WHEN @Order2 = 'Middlename-DESC' THEN Middlename
WHEN @Order2 = 'Lastname-DESC' THEN Lastname
WHEN @Order2 = 'EmployeeType-DESC' THEN EmployeeType
WHEN @Order2 = 'Centername-DESC' THEN Centername
WHEN @Order2 = 'Branchname-DESC' THEN Branchname
WHEN @Order2 = 'Email-DESC' THEN Email
WHEN @Order1 = 'CompletedDate-DESC' THEN CONVERT(VARCHAR, CRecordFinish, 120)
ELSE 'nosortd'
END DESC ,
CASE WHEN @Order3 = 'Username' THEN Username
WHEN @Order3 = 'Firstname' THEN Firstname
WHEN @Order3 = 'Middlename' THEN Middlename
WHEN @Order3 = 'Lastname' THEN Lastname
WHEN @Order3 = 'EmployeeType' THEN EmployeeType
WHEN @Order3 = 'Centername' THEN Centername
WHEN @Order3 = 'Branchname' THEN Branchname
WHEN @Order3 = 'Email' THEN Email
WHEN @Order1 = 'CompletedDate' THEN CONVERT(VARCHAR, CRecordFinish, 120)
ELSE 'nosorte'
END ASC ,
CASE WHEN @Order3 = 'Username-DESC' THEN Username
WHEN @Order3 = 'Firstname-DESC' THEN Firstname
WHEN @Order3 = 'Middlename-DESC' THEN Middlename
WHEN @Order3 = 'Lastname-DESC' THEN Lastname
WHEN @Order3 = 'EmployeeType-DESC' THEN EmployeeType
WHEN @Order3 = 'Centername-DESC' THEN Centername
WHEN @Order3 = 'Branchname-DESC' THEN Branchname
WHEN @Order3 = 'Email-DESC' THEN Email
WHEN @Order1 = 'CompletedDate-DESC' THEN CONVERT(VARCHAR, CRecordFinish, 120)
ELSE 'nosortf'
END DESC

SET ROWCOUNT 0

/* Retrieve the recordset for the Page */
SELECT CASE WHEN @Fmt = 0
THEN '<a href="ad-useredit.asp?UserID=' + CAST(U.UserID as VARCHAR)+
'" target="_blank">' + CAST(U.UserID AS VARCHAR) + '</a>'
ELSE CAST (U.UserID AS VARCHAR)
END as UserID ,
FirstName ,
Middlename as Mi ,
LastName ,
CASE WHEN LEN(C.Centername) > 20 Then STUFF(C.Centername, 20, 0, '<wbr>')
ELSE C.Centername
END AS Centername,
CASE WHEN LEN(B.Branchname) > 100 Then STUFF(B.Branchname, 10, 0, '<wbr>')
ELSE REPLACE(B.Branchname, '/', '/<wbr>')
END AS Branchname,
Username,
Email,
CourseIteration AS FY ,
CRecordID as Certificate ,
CASE WHEN @Fmt = 0
THEN
CASE WHEN CRecordFinish IS NULL THEN ''
ELSE REPLACE(CONVERT(varchar, CRecordFinish, 0), ' ', ' ') END
ELSE
CASE WHEN CRecordFinish IS NULL THEN ''
ELSE CONVERT(varchar, CRecordFinish, 0) END

END AS [Completed],
CASE WHEN @Fmt = 0
THEN
CASE WHEN Duration is NULL THEN ''
WHEN Duration > 120 THEN '> 2 hr'
ELSE CAST(Duration/60 AS VARCHAR) + ' hh, ' +
CAST(Duration-(Duration/60)*60 AS VARCHAR) + ' mm'
END
ELSE
CASE WHEN Duration is NULL THEN ''
WHEN Duration > 120 THEN '> 2 hr'
ELSE CAST(Duration/60 AS VARCHAR) + ' hh, ' +
CAST(Duration-(Duration/60)*60 AS VARCHAR) + ' mm'
END
END AS Duration,
CASE CRAcknowledge
WHEN 0 THEN 'Deny'
WHEN 1 THEN 'Accept'
ELSE 'None'
END AS Accept ,
U.EmployeeType as [Type],
CASE U.ISSO WHEN 1 THEN 'Yes' ELSE 'No' END AS ISSO ,
CASE U.Offline WHEN 1 THEN 'Yes' ELSE 'No' END AS Offline ,
CASE U.Inactive WHEN 1 THEN 'Active' WHEN 2 THEN 'Inactive' ELSE 'Deleted' END AS Status ,
CS.Lang AS Lg

FROM #TempRowLookup R
INNER JOIN dbo.Users U ON U.UserID = R.UserID
LEFT OUTER JOIN dbo.CourseRecords CR
ON CR.UserID = U.UserID
AND CR.CourseID = @CourseID
AND CR.CRecordFinish BETWEEN @CompleteStart AND @CompleteFinish
LEFT OUTER JOIN dbo.Courses CS ON CS.CourseID = CR.CourseID
LEFT OUTER JOIN dbo.Branches B on U.BranchID=B.BranchID
LEFT OUTER JOIN dbo.Centers C on U.CenterID=C.CenterID

WHERE R.RowID >= @PageSize * @CurrentPage
AND R.RowID < @PageSize * (@CurrentPage + 1)
OR @PageSize = 0 -- Return all rows in recordset
ORDER BY R.RowID

DROP TABLE #TempRowLookup


RETURN(0)
Go to Top of Page

ehorn
Master Smack Fu Yak Hacker

1632 Posts

Posted - 2004-01-07 : 11:35:07
How about considering a composite index on rowid and userid ?
It seems that this is a covered query and would stand to gain from a composite index..

Would another option be to perform a select into #table with identity. then apply your paging criteria?

EDIT:

Also, would it be possible to make rowid and userid smallint or do you expect more than 32,767 rows or users?

This will reduce the table size and create a more narrow index.

PS. I thought this was a good article.
http://www.mssqlcity.com/Tips/tipInd.htm
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2004-01-07 : 11:38:33
Well it looks like your predicate is only on ROWID...

And why are you building a temp table that has 3 LEFT OUTER JOINS, only to do again against the TEMP?

Why not denorm the whole thing up front?



Brett

8-)
Go to Top of Page

SamC
White Water Yakist

3467 Posts

Posted - 2004-01-07 : 12:10:35

I was thinking that the PRIMARY KEY on RowID would create a clustered index which should be good for retrieving the page of rows. I understand that this would rule out any benefit of creating a COMPOSITE on (RowID, UserID).

Yes, there could be 32K+ rows, so INT is needed. I don't expect it to exceed more than 1000 rows for 99.99% of the queries, but I don't want it to break if a user doesn't apply a filter.

To me, it seems adding an index on UserID alone will do, but paging algorithms are so common, this should have a canned answer. It's interesting that the articles on paging completely overlook the issue of indexing.

This leads to Brett's question which made me go back to verify my sanity (I'll leave out what I found there) - the LEFT JOINS are needed on the insert / select to enable the WHERE and ORDER BY to perform - a critical matter when inserting UserID in the proper order in the temporary table. Later, when the records are retrieved for the specific page, the LEFT JOINS are needed to retrieve the actual values in the recordset.

An alternative would be to insert all the data in the temp table - then no joins are needed to retrieve the data. It's a close call as to which performs better and may depend on the total rows and number of columns. I usually opt to choose the solution that moves less data.

This raises a design issue - whether the proc would perform better if inserting additional IDs in the temporary table could eliminate the JOINs in the 2nd SELECT statement, but I say: I've got bigger fish to fry and pass the margaritas.

Sam
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2004-01-07 : 12:16:59
"smell of the shrimp, they're beginning to boil...oooo..wasted away..."


Anyway, why not try the index intersection thing...

but the less time you have to go to the tables the better....

And just one point about the 4 guys example...it's sorted by cost...

which could be different every time, if they have a lot of the same cost...such that a return to page 1 over and over may show different rows...



Brett

8-)
Go to Top of Page

SamC
White Water Yakist

3467 Posts

Posted - 2004-01-07 : 12:26:50
"index intersection thing..." - could you be any more cryptic? OH! That index intersection thing !

Per the 4 Guys approach, when there's only 1 item to sort on, then the approach in Mr. Cross Join's blog has a temporary-table-free solution. As soon as the sort can occur on any of several items, you're hosed.

Sam
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2004-01-07 : 12:52:42
quote:
Originally posted by SamC

"index intersection thing..." - could you be any more cryptic? OH! That index intersection thing !



Sam,

You only started about a 3 page thread about that...forget so soon?

http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=30090&SearchTerms=Index,Intersection



Brett

8-)
Go to Top of Page

SamC
White Water Yakist

3467 Posts

Posted - 2004-01-07 : 14:24:02
How bout that? No, I'd forgotten.

Ya know, that thread cleared the fog a little, then it started to rain. Indexes are an art not a science.

Sam
Go to Top of Page

rb
Starting Member

1 Post

Posted - 2004-01-08 : 00:58:16
Hi All,
I am new to .Net development but for recordset pagingin in ASP I have been using the approach given at following URL with a lot of success.

http://www.adopenstatic.com/experiments/recordsetpaging.asp

Regards,
RB
Go to Top of Page
   

- Advertisement -