Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
akshayg01
Starting Member
1 Post |
 Posted - 2009-09-22 : 03:05:03
|
| I need to design a system which will process hundreds of source files (different format) and convert to one target files. There should be two interfaces 1. Command line2. web user interface Command line interface is to run the transform job though batch and web user interface is to define the format of source file and mapping details of source file to target file i.e. one time job for each source file.All the source files are fixed width or delimited files. What is the correct approach? Should I create one stage table for each source file to the stage data? Should I create one stage table on runtime when user will define of layout (Most are fixed width mainframe files) of source file?Should I create only one generic table, containing around 100 columns all varchar type? I am looking for the best approach to design the system. Performance is very critical for this app. There are hundreds of files and we need to transform all the files daily within certain time. Thanks in AdvanceAkshay |
|
|
GilaMonster
Master Smack Fu Yak Hacker
4507 Posts |
Posted - 2009-09-22 : 08:01:59
|
| Have you looked at SSIS? It's designed as an ETL tool, so it should be possible to use it for this.--Gail ShawSQL Server MVP |
 |
|
|
Lumbago
Norsk Yak Master
3271 Posts |
Posted - 2009-09-22 : 11:00:28
|
| Are we talking flat file data format conversion here? In that case I'd really consider not using a database at all...- Lumbago |
|
|
|
|
|
|
Design system to transform source files to target.