Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
Bex
Aged Yak Warrior
580 Posts |
 Posted - 2009-03-16 : 07:59:23
|
| I have to increase the amount of test data in a database. The way I am doing this is to select each tables data back into itself, and transform an IDENTITY field (to be unique) and to change the store number to a new one.The database consists of 18 tables, but have been manually partitioned across dates - currently 2 weeks worth (so there are 14 sets of tables - one for each day, and each set of tables consists of 1 header and 17 child tables).The SSIS consist of a parent package, that does a few calculations and then populates all 14 header tables in a foreach loop. It then calls 17 child packages (of which only 7/8 are enabled as not all are relevant). Each child package populates the child tables (17x14 of them) in a foreach loop. There is not a ridiculous amount of data (about 30,000 transactions over 14 days).When I had this running in a single package, this was flying through, (but would then stop - see thread ''SSIS package just cancels itself'').Now it is crawling at a snails pace. I think the data has so far gone through 2 iterations:Began with 4 stores1st iteration = 4+42nd Iteration = 8+8And this has been running for about an hour and a half???I have deployed the packages to Integration services via file system. They connect to a remote server (using configuration file) and are running locally on my box (4GB, SQL Server 2005, Windows Server 2003). This is insane, now it works, but I am supposed to be extrapolating from 4 stores to 2000. So far, I have 16? This is not right.Any suggestions as to what I am doing wrong (I am fairly new to ssis)? Are there any other ways to approach this?Should I perhaps copy to a flat file, do transformations and import back in? when I used bcp to copy a ridiculous number of records (millions not a measly few thousand) to a flat file, this took a matter of minutes?Hearty head pats |
|
|
Bex
Aged Yak Warrior
580 Posts |
Posted - 2009-03-16 : 08:11:47
|
| Could it be a network issue? If I am working on my local box?Hearty head pats |
 |
|
|
basicconfiguration
Constraint Violating Yak Guru
358 Posts |
Posted - 2010-03-08 : 16:06:10
|
| Did you ever fix this issue? |
|
|
|
|
|
|
|
|
Why so slow???