Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
jpotucek
Constraint Violating Yak Guru
273 Posts |
 Posted - 2009-11-05 : 10:00:07
|
| MS SQL Server 2005 enterprise edition Our App Dev group started a series of large insert jobs that are inserting millions of rows into one particular database from flat files. been running for days now... I left the database in Full recovery mode and sized the Transaction log so that it would not have to continually grow as the inserts were happening. (TLOG is set to grow in 10% increments) I'm just wondering how others would have handled this ? Should I have put the DB in SIMPLE recovery mode for the duration of the intial load (for performance sake) ???We will be restoring the DB and running the initial again next week (in dev) before doing the final initial load into production sometime next month.. I'd just like to make sure that I'm not slowing down the process.. any suggestions? |
|
|
AndrewMurphy
Master Smack Fu Yak Hacker
2916 Posts |
Posted - 2009-11-05 : 10:32:42
|
| 1. simple recovery YES2. bcp better tool than large-insert jobs3. batching data better still.4. app-dev being shot - for serious consideration, especially if actioned without reference to DBA's. |
 |
|
|
jpotucek
Constraint Violating Yak Guru
273 Posts |
Posted - 2009-11-05 : 10:36:49
|
| Simple recovery mode makes sense.. They are batching the data and I believe that they are running the vendors import program because there is some transformation that has to happen as these flat files are from the mainframe and require a lot of verifying for dups, bad data etc... |
|
|
|
AndrewMurphy
Master Smack Fu Yak Hacker
2916 Posts |
Posted - 2009-11-06 : 10:40:54
|
| SQLServer data import can do data transforms/verifications.Some experimentation on a small data load/area might give a strong hint if switching technical horses would be worthwhile in the end. |
|
|
|
|
|
|
proper DB model for large number of inserts