Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 General SQL Server Forums
 Database Design and Application Architecture
 column store databases

Author  Topic 

jezemine
Master Smack Fu Yak Hacker

2886 Posts

Posted - 2010-06-16 : 19:06:03
Does anyone have any experience with column store databases? http://en.wikipedia.org/wiki/Column-oriented_DBMS

I have a looming situation where I will be generating up to 3 million rows of numbers per minute, and need to store them. table schema something like this:


create table GrowsFast (
TimeKey int not null
,SegmentID int not null
,Speed tinyint not null
,Error tinyint not null
-- a few other things
,primary key (TimeKey, SegmentID)
)


Much of the data in a given column will be the same, for example each minute, all values of TimeKey will be the same. From minute to minute, the similar SegmentID will appear (there are about 40 million possible values for SegmentID, but only approx 3 m will actually be present any particular minute). Speed can only range from 1 to 75 say. Same for Error. So lots of same values.

So I was thinking may be a column store db may be a good way to go. I've never used one though so though I'd poll for opinions.




elsasoft.org

X002548
Not Just a Number

15586 Posts

Posted - 2010-06-16 : 19:47:26
there is only a couple of was to get millions of rows per minute...bcp and bulk insert

so what is all this nonsense


and, for a key, the cpu clock is only good to .003


Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx

Add yourself!
http://www.frappr.com/sqlteam



Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2010-06-17 : 02:32:39
I reckon the "timekey" is coming in with the data from something with its own Atomic clock Brett!

But peripheral question:

How does one get more accuracy from SYSDATETIME(), as an alternative to GetDate()

select GetDate(), SYSDATETIME()

2010-06-17 07:31:40.737 2010-06-17 07:31:40.7380000

More zeros ... Yummie!
Go to Top of Page

jezemine
Master Smack Fu Yak Hacker

2886 Posts

Posted - 2010-06-17 : 02:50:25
TimeKey is like a unix time, but with minute accuracy only. it's the number of minutes from some arbitrary date in the recent past.


elsasoft.org
Go to Top of Page
   

- Advertisement -