Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
SamC
White Water Yakist
3467 Posts |
Posted - 2004-01-07 : 09:22:25
|
| I'm implementing recordset paging in a stored procedure. There are many sort options and filter options. The usual technique of placing the sorted material in a temporary table like this:CREATE TABLE #MyOrder ( MyID INT IDENTITY (1,1) PRIMARY KEY, UserID INT )After the table is populated, then records N to N+M are extracted for the requested page, and UserID is used to find the needed records. Several joins exist to retrieve the data, and I'm wondering how best to index UserID in the temp table?CREATE UNIQUE CLUSTERED INDEX abc ON #MyOrder (MyID, UserID) -- I'd get rid of the primary key above for thisCREATE UNIQUE INDEX abc ON #MyOrder (UserID)-------------------------------------------------And a separate question:Here's an observation I made that I wonder if anyone can confirm for me. I once ran some tests populating a similar table and found that it ran much faster if I populated the table first, then created the index before using the data. It makes some sense if you think about it. Population is atomic (single insert), the indexing of the data could be faster if it was done in a single swoop, after the table was filled.Sam |
|
|
X002548
Not Just a Number
15586 Posts |
Posted - 2004-01-07 : 10:13:41
|
| Sam,Why a temp table?It'll disappear after the sproc ends...So why put more data in than is required...seems like you'll only be able to present 1 page at a time anyway....And yes bilding the indexes after the fact sometimes make sense...But can you give use more background on what your doing?Brett8-) |
 |
|
|
SamC
White Water Yakist
3467 Posts |
Posted - 2004-01-07 : 10:56:32
|
Hi Brett,The direction I've taken to retrieve paged data for web display is derived from articles on 4-Guys, Mr. Cross Join's BLOG and posts here on SQLTEAM.There are several methods to page resultset data, but the consensus on optimal recordset paging (in SQL 2000) tasks the stored procedure to build a query for the result set, then inserting all the data (or a unique identifier) into a temporary table with a sequence column built using IDENTITY(1,1). The sequenced record (column) numbers are then used to easily find the *page* which is a starting row number, and a number of rows following the starting point.In my case, the query has multiple filters, and 3 ordering levels which can be either ascending or descending. This solution is pretty good for a code-it-yourself solution, but paging will be a big part of the web future so I like to think Yukon will have better support for paging. (anyone have info on this?) Paging data ought to be easier than what we do in T-SQL.The temporary table is a good choice if there is a possability of it being used by more than 1 user at a time. I might convert the temporary table to a local table variable - once everything is debuged.I've included the code to show how complex the whole paging process becomes. I think the idea for ascending / descending ORDER BY came from Jeff.I may draw some flack about the insertion of HTML in the resulting data, but hey, this is preproduction testing, not commercial stuff yet.Sam/* For paging, we retrieve the selected UserIDs, in order, into a temporary table */CREATE TABLE #TempRowLookup ( RowID INT IDENTITY(0,1) PRIMARY KEY, -- We can retrieve any sequence of ordered rows for a specific page using RowID UserID INT )/* We don't need records beyond the end of page */DECLARE @RowLimit INTSET @RowLimit = @PageSize * (@CurrentPage + 1)SET ROWCOUNT @RowLimitINSERT INTO #TempRowLookup (UserID) SELECT U.UserID FROM dbo.Users U LEFT OUTER JOIN dbo.CourseRecords CR ON CR.UserID = U.UserID AND CR.CourseID = @CourseID AND CR.CRecordFinish BETWEEN @CompleteStart AND @CompleteFinish LEFT OUTER JOIN dbo.Courses CS ON CS.CourseID = CR.CourseID LEFT OUTER JOIN dbo.Branches B on U.BranchID=B.BranchID LEFT OUTER JOIN dbo.Centers C on U.CenterID=C.CenterID WHERE U.ClientID = @ClientID AND (@Email IS NULL OR Email LIKE @Email) AND (@Firstname IS NULL OR Firstname LIKE @Firstname) AND (@Middlename IS NULL OR Middlename LIKE @Middlename) AND (@Lastname IS NULL OR Lastname LIKE @Lastname) AND (@Username IS NULL OR Username LIKE @Username) AND (@EmployeeType IS NULL OR EmployeeType LIKE @EmployeeType) AND (@CenterID IS NULL OR U.CenterID = @CenterID OR U.CenterID = 0) AND (@BranchID IS NULL OR U.BranchID = @BranchID) AND (@Wildcard IS NULL OR B.Branchname LIKE @Branchname) AND (@Offline IS NULL OR U.Offline = @Offline) AND (@InactiveMask IS NULL OR (U.Inactive & @InactiveMask <> 0)) AND (@Completed IS NULL OR (@Completed=0 AND CRecordFinish IS NULL) OR (@Completed=1 AND CRecordFinish BETWEEN @CompleteStart AND @CompleteFinish) ) AND (@Acknowledge IS NULL OR (@Acknowledge=1 AND CRAcknowledge=1) OR (@Acknowledge=0 AND (CRAcknowledge = 0 or CRAcknowledge = -1))) ORDER BY CASE WHEN @Order1 = 'Username' THEN Username WHEN @Order1 = 'Firstname' THEN Firstname WHEN @Order1 = 'Middlename' THEN Middlename WHEN @Order1 = 'Lastname' THEN Lastname WHEN @Order1 = 'EmployeeType' THEN EmployeeType WHEN @Order1 = 'Centername' THEN Centername WHEN @Order1 = 'Branchname' THEN Branchname WHEN @Order1 = 'Email' THEN Email WHEN @Order1 = 'CompletedDate' THEN CONVERT(VARCHAR, CRecordFinish, 120) ELSE 'nosorta' END ASC , CASE WHEN @Order1 = 'Username-DESC' THEN Username WHEN @Order1 = 'Firstname-DESC' THEN Firstname WHEN @Order1 = 'Middlename-DESC' THEN Middlename WHEN @Order1 = 'Lastname-DESC' THEN Lastname WHEN @Order1 = 'EmployeeType-DESC' THEN EmployeeType WHEN @Order1 = 'Centername-DESC' THEN Centername WHEN @Order1 = 'Branchname-DESC' THEN Branchname WHEN @Order1 = 'Email-DESC' THEN Email WHEN @Order1 = 'CompletedDate-DESC' THEN CONVERT(VARCHAR, CRecordFinish, 120) ELSE 'nosortb' END DESC , CASE WHEN @Order2 = 'Username' THEN Username WHEN @Order2 = 'Firstname' THEN Firstname WHEN @Order2 = 'Middlename' THEN Middlename WHEN @Order2 = 'Lastname' THEN Lastname WHEN @Order2 = 'EmployeeType' THEN EmployeeType WHEN @Order2 = 'Centername' THEN Centername WHEN @Order2 = 'Branchname' THEN Branchname WHEN @Order2 = 'Email' THEN Email WHEN @Order1 = 'CompletedDate' THEN CONVERT(VARCHAR, CRecordFinish, 120) ELSE 'nosortc' END ASC , CASE WHEN @Order2 = 'Username-DESC' THEN Username WHEN @Order2 = 'Firstname-DESC' THEN Firstname WHEN @Order2 = 'Middlename-DESC' THEN Middlename WHEN @Order2 = 'Lastname-DESC' THEN Lastname WHEN @Order2 = 'EmployeeType-DESC' THEN EmployeeType WHEN @Order2 = 'Centername-DESC' THEN Centername WHEN @Order2 = 'Branchname-DESC' THEN Branchname WHEN @Order2 = 'Email-DESC' THEN Email WHEN @Order1 = 'CompletedDate-DESC' THEN CONVERT(VARCHAR, CRecordFinish, 120) ELSE 'nosortd' END DESC , CASE WHEN @Order3 = 'Username' THEN Username WHEN @Order3 = 'Firstname' THEN Firstname WHEN @Order3 = 'Middlename' THEN Middlename WHEN @Order3 = 'Lastname' THEN Lastname WHEN @Order3 = 'EmployeeType' THEN EmployeeType WHEN @Order3 = 'Centername' THEN Centername WHEN @Order3 = 'Branchname' THEN Branchname WHEN @Order3 = 'Email' THEN Email WHEN @Order1 = 'CompletedDate' THEN CONVERT(VARCHAR, CRecordFinish, 120) ELSE 'nosorte' END ASC , CASE WHEN @Order3 = 'Username-DESC' THEN Username WHEN @Order3 = 'Firstname-DESC' THEN Firstname WHEN @Order3 = 'Middlename-DESC' THEN Middlename WHEN @Order3 = 'Lastname-DESC' THEN Lastname WHEN @Order3 = 'EmployeeType-DESC' THEN EmployeeType WHEN @Order3 = 'Centername-DESC' THEN Centername WHEN @Order3 = 'Branchname-DESC' THEN Branchname WHEN @Order3 = 'Email-DESC' THEN Email WHEN @Order1 = 'CompletedDate-DESC' THEN CONVERT(VARCHAR, CRecordFinish, 120) ELSE 'nosortf' END DESC SET ROWCOUNT 0/* Retrieve the recordset for the Page */SELECT CASE WHEN @Fmt = 0 THEN '<a href="ad-useredit.asp?UserID=' + CAST(U.UserID as VARCHAR)+ '" target="_blank">' + CAST(U.UserID AS VARCHAR) + '</a>' ELSE CAST (U.UserID AS VARCHAR) END as UserID , FirstName , Middlename as Mi , LastName , CASE WHEN LEN(C.Centername) > 20 Then STUFF(C.Centername, 20, 0, '<wbr>') ELSE C.Centername END AS Centername, CASE WHEN LEN(B.Branchname) > 100 Then STUFF(B.Branchname, 10, 0, '<wbr>') ELSE REPLACE(B.Branchname, '/', '/<wbr>') END AS Branchname, Username, Email, CourseIteration AS FY , CRecordID as Certificate , CASE WHEN @Fmt = 0 THEN CASE WHEN CRecordFinish IS NULL THEN '' ELSE REPLACE(CONVERT(varchar, CRecordFinish, 0), ' ', ' ') END ELSE CASE WHEN CRecordFinish IS NULL THEN '' ELSE CONVERT(varchar, CRecordFinish, 0) END END AS [Completed], CASE WHEN @Fmt = 0 THEN CASE WHEN Duration is NULL THEN '' WHEN Duration > 120 THEN '> 2 hr' ELSE CAST(Duration/60 AS VARCHAR) + ' hh, ' + CAST(Duration-(Duration/60)*60 AS VARCHAR) + ' mm' END ELSE CASE WHEN Duration is NULL THEN '' WHEN Duration > 120 THEN '> 2 hr' ELSE CAST(Duration/60 AS VARCHAR) + ' hh, ' + CAST(Duration-(Duration/60)*60 AS VARCHAR) + ' mm' END END AS Duration, CASE CRAcknowledge WHEN 0 THEN 'Deny' WHEN 1 THEN 'Accept' ELSE 'None' END AS Accept , U.EmployeeType as [Type], CASE U.ISSO WHEN 1 THEN 'Yes' ELSE 'No' END AS ISSO , CASE U.Offline WHEN 1 THEN 'Yes' ELSE 'No' END AS Offline , CASE U.Inactive WHEN 1 THEN 'Active' WHEN 2 THEN 'Inactive' ELSE 'Deleted' END AS Status , CS.Lang AS Lg FROM #TempRowLookup R INNER JOIN dbo.Users U ON U.UserID = R.UserID LEFT OUTER JOIN dbo.CourseRecords CR ON CR.UserID = U.UserID AND CR.CourseID = @CourseID AND CR.CRecordFinish BETWEEN @CompleteStart AND @CompleteFinish LEFT OUTER JOIN dbo.Courses CS ON CS.CourseID = CR.CourseID LEFT OUTER JOIN dbo.Branches B on U.BranchID=B.BranchID LEFT OUTER JOIN dbo.Centers C on U.CenterID=C.CenterID WHERE R.RowID >= @PageSize * @CurrentPage AND R.RowID < @PageSize * (@CurrentPage + 1) OR @PageSize = 0 -- Return all rows in recordset ORDER BY R.RowIDDROP TABLE #TempRowLookupRETURN(0) |
 |
|
|
ehorn
Master Smack Fu Yak Hacker
1632 Posts |
Posted - 2004-01-07 : 11:35:07
|
| How about considering a composite index on rowid and userid ? It seems that this is a covered query and would stand to gain from a composite index..Would another option be to perform a select into #table with identity. then apply your paging criteria?EDIT:Also, would it be possible to make rowid and userid smallint or do you expect more than 32,767 rows or users? This will reduce the table size and create a more narrow index.PS. I thought this was a good article. http://www.mssqlcity.com/Tips/tipInd.htm |
 |
|
|
X002548
Not Just a Number
15586 Posts |
Posted - 2004-01-07 : 11:38:33
|
| Well it looks like your predicate is only on ROWID...And why are you building a temp table that has 3 LEFT OUTER JOINS, only to do again against the TEMP?Why not denorm the whole thing up front?Brett8-) |
 |
|
|
SamC
White Water Yakist
3467 Posts |
Posted - 2004-01-07 : 12:10:35
|
| I was thinking that the PRIMARY KEY on RowID would create a clustered index which should be good for retrieving the page of rows. I understand that this would rule out any benefit of creating a COMPOSITE on (RowID, UserID).Yes, there could be 32K+ rows, so INT is needed. I don't expect it to exceed more than 1000 rows for 99.99% of the queries, but I don't want it to break if a user doesn't apply a filter.To me, it seems adding an index on UserID alone will do, but paging algorithms are so common, this should have a canned answer. It's interesting that the articles on paging completely overlook the issue of indexing.This leads to Brett's question which made me go back to verify my sanity (I'll leave out what I found there) - the LEFT JOINS are needed on the insert / select to enable the WHERE and ORDER BY to perform - a critical matter when inserting UserID in the proper order in the temporary table. Later, when the records are retrieved for the specific page, the LEFT JOINS are needed to retrieve the actual values in the recordset.An alternative would be to insert all the data in the temp table - then no joins are needed to retrieve the data. It's a close call as to which performs better and may depend on the total rows and number of columns. I usually opt to choose the solution that moves less data.This raises a design issue - whether the proc would perform better if inserting additional IDs in the temporary table could eliminate the JOINs in the 2nd SELECT statement, but I say: I've got bigger fish to fry and pass the margaritas.Sam |
 |
|
|
X002548
Not Just a Number
15586 Posts |
Posted - 2004-01-07 : 12:16:59
|
| "smell of the shrimp, they're beginning to boil...oooo..wasted away..."Anyway, why not try the index intersection thing...but the less time you have to go to the tables the better....And just one point about the 4 guys example...it's sorted by cost...which could be different every time, if they have a lot of the same cost...such that a return to page 1 over and over may show different rows...Brett8-) |
 |
|
|
SamC
White Water Yakist
3467 Posts |
Posted - 2004-01-07 : 12:26:50
|
"index intersection thing..." - could you be any more cryptic? OH! That index intersection thing !Per the 4 Guys approach, when there's only 1 item to sort on, then the approach in Mr. Cross Join's blog has a temporary-table-free solution. As soon as the sort can occur on any of several items, you're hosed.Sam |
 |
|
|
X002548
Not Just a Number
15586 Posts |
|
|
SamC
White Water Yakist
3467 Posts |
Posted - 2004-01-07 : 14:24:02
|
| How bout that? No, I'd forgotten.Ya know, that thread cleared the fog a little, then it started to rain. Indexes are an art not a science.Sam |
 |
|
|
rb
Starting Member
1 Post |
|
|
|
|
|
|
|