On Mon, Mar 2= 6, 2012 at 4:10 AM, aaron morton <aaron@thelastpickle.com> wrote:

There is a great deal of utility in bee= n able to derive the set of possible row keys for a date range on the clien= t side. So I would try to carve up the time slices with respect to the time= rather than the amount of data in them. This may not be practical but I th= ink it's very useful.=A0

Say you are storing the raw time series facts in the Fact CF= , and the row key is something like <source:datetime> (may want to ad= d a bucket size see below) and the column name is the <isotimestamp>.= The data source also has a bucket size stored something, such as hourly, d= aily, month.=A0

For an hourly bucket source, the datetime in the r= ow keys is something like "2012-01-02T13:00" (one for each hour) = for a daily it's something like "2012-01-02T00:00" . You can = then work out the set of possible keys in a date range and perform multi se= lects against those keys until you have all the data.=A0

If you change the bucketing scheme for a data source yo= u need to keep a history so you can work out which keys may exist. That may= be a huge pain. As an alternative create a custom secondary, as you discus= sed, of all the row keys for the data source. But continue to use a consist= ent time based method for partitioning time ranges if possible.=A0

=A0=A0

Hope that helps.=A0

<= div>

-----------------

Aaron Morton

Freelance Deve= loper

@aaronmorton

http://www.thelastpickle.com

On 24/03/2012, at 3:22 AM, Jim A= ncona wrote:

I'm dealing with a simi= lar issue, with an additional complication. We are collecting time series d= ata, and the amount of data per time period varies greatly. We will collect= and query event data by account, but the biggest account will accumulate a= bout 10,000 times as much data per time period as the median account. So fo= r the median account I could put multiple years of data in one row, while f= or the largest accounts I don't want to put more one day's worth in= a row. If I use a uniform bucket size of one day (to accomodate the larges= t accounts) it will make for rows that are too short for the vast majority = of accounts--we would have to read thirty rows to get a month's worth o= f data. One obvious approach is to set a maximum row size, that is, write d= ata in a row until it reaches a maximum length, then start a new one. There= are two things that make that harder than it sounds:

There's no efficient way to count columns in a Cassandra row in ord= er to find out when to start a new one.=A0

Row keys aren't searchable. So I need to be able to construct or lo= ok up the key to each row that contains a account's data. (Our data wil= l be in reverse date order.)

Possible solutions:

Cassandra counter columns are an efficient way to keep counts

I could have a "directory" row that contains pointers to the = rows that contain an account data

(I could probably combine the row directory and the column counter = into a single counter column family, where the column name is the row key a= nd the value is the counter.) A naive solution would require reading the di= rectory before every read and the counter before every write--caching could= probably help with that. So this approach would probably lead to a reasona= ble solution, but it's liable to be somewhat complex. Before I go much = further down this path, I thought I'd run it by this group in case some= one can point out a more clever solution.

Thanks,
Jim
On Thu, Mar 22, 2012 at = 5:36 PM, Alexandru Sicoe <adsicoe@gmail.com> wrote:
Thanks Aaron, I'll lower the time bucket, see how it goes.

Cheer= s,
Alex

On Thu, Mar 22, 2012 at 10:07 PM, aaron morton <aaron@thelastpickle.= com> wrote:

Will add= ing a few tens of wide rows like this every day cause me problems on the lo= ng term? Should I consider lowering the time bucket?

IMHO yeah, yup, ya and yes.

From experience I am a bit reluctant to create too = many rows because I see that reading across multiple rows seriously affects= performance. Of course I will use map-reduce as well ...will it be signifi= cantly affected by many rows?

Don't think it would make too much difference.=A0
range slice used by map-reduce will find the first row in the batch= and then step through them.

Cheers

-----------------
Aaron Morton
Freelance Deve= loper
@aaronmorton
http://www.thelastpickle.com

On 22/03/2012, at 11:43 PM, Alexandru Sicoe w= rote:

Hi guys,

Based on what you = are saying there seems to be a tradeoff that developers have to handle betw= een:

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 "keep your rows under a certain size" vs &q= uot;keep data that's queried together, on disk together"

How would you handle this tradeoff in my case:

I monitor about = 40.000 independent timeseries streams of data. The streams have highly vari= able rates. Each stream has its own row and I go to a new row every 28 hrs.= With this scheme, I see several tens of rows reaching sizes in the million= s of columns within this time bucket (largest I saw was 6.4 million). The s= izes of these wide rows are around 400MBytes (considerably > than 60MB)<= br>
Will adding a few tens of wide rows like this every day cause me proble= ms on the long term? Should I consider lowering the time bucket?

Fro= m experience I am a bit reluctant to create too many rows because I see tha= t reading across multiple rows seriously affects performance. Of course I w= ill use map-reduce as well ...will it be significantly affected by many row= s?

Cheers,
Alex

On Tue, Mar 20, 2012 = at 6:37 PM, aaron morton <aaron@thelastpickle.com> wro= te:

The read= s are only fetching slices of 20 to 100 columns max at a time from the row = but if the key is planted on one node in the cluster I am concerned about t= hat node getting the brunt of traffic.

What RF are you using, how many nodes are in the cluster, what CL do = you read at ?

If you have lots of nodes that are in diff= erent racks the NetworkTopologyStrategy will do a better job of distributin= g read load than the SimpleStrategy. The DynamicSnitch can also result dist= ribute load, see cassandra yaml for it's configuration.=A0

I thought about breaking= the column data into multiple different row keys to help distribute throug= hout the cluster but its so darn handy having all the columns in one key!!<= br>
If you have a row that will continually grow it is a goo= d idea to partition it in some way. Large rows can slow things like compact= ion and repair down. If you have something above 60MB it's starting to = slow things down. Can you partition by a date range such as month ?

Large rows are also a little slower to query from
=
http://thelastpickle.com/2011/07/04/Cassandra-Query-Plan= s/

If most reads are only pulling 20 to 100 columns at a t= ime are there two workloads ? Is it possible store just these columns in a = separate row ? If you understand how big a row may get may be able to use t= he row cache to improve performance. =A0

Cheers

-----------------
Aaron Morton
Freelance Deve= loper
@aaronmorton
http://www.thelastpickle.com

On 20/03/2012, at 2:05 PM, Blake Starkenburg wrote:

=
I have a row key which is now up to 125,000 colum= ns (and anticipated to grow), I know this is a far-cry from the 2-billion c= olumns a single row key can store in Cassandra but my concern is the amount= of reads that this specific row key may get compared to other row keys. Th= is particular row key houses column data associated with one of the more po= pular areas of the site. The reads are only fetching slices of 20 to 100 co= lumns max at a time from the row but if the key is planted on one node in t= he cluster I am concerned about that node getting the brunt of traffic.

I thought about breaking the column data into multiple different row ke= ys to help distribute throughout the cluster but its so darn handy having a= ll the columns in one key!!

key_cache is enabled but row cache is di= sabled on the column family.

Should I be concerned going forward? Any particular advice on large wid= e rows?

Thanks!