Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 General SQL Server Forums
 Database Design and Application Architecture
 Scaling up SQL Server Full-text index population

Author  Topic 

mohdowais
Sheikh of Yak Knowledge

1456 Posts

Posted - 2011-07-07 : 04:09:54
Wow, it's been a while since I've posted anything here. Good to see a lot of the regulars are still here. Anyway, I have an issue that I could use ideas for.

We have a table that stores attachments in a varbinary column and is fulltext-indexed. It currently has over 5 million rows and we are adding about 10,000 records every day.

We have no problems with the full-text search but we really are struggling with the full-text population for this table now. Incremental indexing is done daily and takes a few hours, but the full rebuilds every month are just killing the server. The server has a 4 core processor with 12 GB RAM, so isn't exactly a wimp, but the full population easily takes 2 days and maxes out the CPU during this period, causing queries to fail randomly during this period.

How would you scale up/scale out full-text population in this case? I have considered the following, let me know what you think of them or if you have any better ideas:

- Add more CPU resources (not sure how I can prevent the fdhost from stealing all the additional CPU resources though)
- Outsource full-text population to another server and import the completed catalog
- Run full-text queries directly on the other server (using a linked server)
- Use indexed views to partition the data by year and query only the necessary years

Note: I haven't tried any of these yet, I'm scouting for ideas at the moment. Any input would be appreciated. Thanks

OS

OS

mohdowais
Sheikh of Yak Knowledge

1456 Posts

Posted - 2011-07-10 : 05:07:05
For anyone looking for a potential solution, see this thread here: http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=162690. In a nutshell, it seems that REBUILD-ing your fulltext catalog is not necessary, instead you should run a REORGANIZE.

OS
Go to Top of Page
   

- Advertisement -