Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
cohenoshri
Starting Member
2 Posts |
 Posted - 2004-06-17 : 16:57:48
|
i have quite the problem,i am receving a csv file containing roughly 2 - 2.8 million records from a 3rd party.the csv file contains a primary key and 5 other varchar fields, those varchar fields have data that repeats itself across the file.Each one of those fields have their own table w/ a primary key.what i need to do is to read line by line from the csv and get the primary key of each field then insert into one table.the problem that arises is that the insert takes about 7 days of constant processing to insert all the data.i need to repeat this process on a monthly basis.any suggestions on ho i can speed this up???Thank You |
|
|
tkizer
Almighty SQL Goddess
38200 Posts |
Posted - 2004-06-17 : 17:00:00
|
| What method are you using? Have you considered bcp?Tara |
 |
|
|
derrickleggett
Pointy Haired Yak DBA
4184 Posts |
Posted - 2004-06-17 : 17:43:25
|
| You might want to post some sample data, the structure, and explain what you're trying to do here with examples.MeanOldDBAderrickleggett@hotmail.comWhen life gives you a lemon, fire the DBA. |
|
|
|
cohenoshri
Starting Member
2 Posts |
Posted - 2004-06-18 : 16:36:17
|
| ok here is a sample row:"201653760","201654015","US","UNITED STATES","VIRGINIA","GORDONSVILLE","SOUTHERNSTATESCOOPERATIVE,INC" |
|
|
|
tkizer
Almighty SQL Goddess
38200 Posts |
Posted - 2004-06-18 : 16:39:42
|
| Have you considered importing the data into a staging table then getting the data to your real tables with T-SQL? So are you using BULK INSERT command? Do you have a cursor in your code? We need some more information...Tara |
|
|
|
derrickleggett
Pointy Haired Yak DBA
4184 Posts |
Posted - 2004-06-18 : 19:07:04
|
| and explain what you're trying to do here with examples.This part is kind of important.MeanOldDBAderrickleggett@hotmail.comWhen life gives you a lemon, fire the DBA. |
|
|
|
|
|
|
|
|
sql server bulk inserts