Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
AskSQLTeam
Ask SQLTeam Question
0 Posts |
 Posted - 2002-07-23 : 08:59:27
|
| James writes "I am looking for a way to do a bulk load of an in memory dataset that I create from parsing a mainframe file. It is usually about 12.5 million rows (I have alot of memory on my server) and I need to get it into a 140 mill row table. I want to do this as rapidly as possible - ie a bulk load interface, but there is not one exposed in .Net framework. I don't want to write the dataset to a file and then bcp, xml has too much overhead and is inefficient in this case, I want to stream from memory the dataset into the database as fast as possible. I know of the irowsetfastload interface, but can't get to it easily from .net - any ideas?" |
|
|
andre
Constraint Violating Yak Guru
259 Posts |
Posted - 2002-07-23 : 09:16:08
|
| BULK INSERT is the fastest way I can think of to insert data into a table in SQL Server.What COM object has the irowsetfastload interface? Have you tried using tlbimp.exe to convert the COM object into an assembly for use in .NET? |
 |
|
|
robvolk
Most Valuable Yak
15732 Posts |
Posted - 2002-07-23 : 09:22:06
|
quote:
I don't want to write the dataset to a file and then bcp, xml has too much overhead and is inefficient in this case
Why use XML? Unless you're only structuring it in memory. In a pinch, you can output the XML to a file and then use an XSL sheet to transform it into a flat text file that bcp or BULK INSERT can use.Considering that the data comes from a mainframe file, why can't you just import that file directly? How much and what kind of parsing do you need to do? Unless the file is in XML format, in which case you can still use the XSL approach. Not to mention that SQL Server 2000 has XML support and can bulk load XML files. |
|
|
|
|
|
|
Bulk loading a dataset from .Net framework into SQL server