cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Alonso <i...@mrcalonso.com>
Subject Re: Cassandra Data Model with Narrow partition
Date Fri, 30 Oct 2015 12:04:34 GMT
Hi Chandra,

Narrow partition is probably your best choice, but you need to bucket data
somehow, otherwise your partitions will soon become unmanageable and you'll
have problems reading them, both because the partitions will become very
big and also because of the tombstones that your expired records will
generate.

In general having a partition that can grow indefinitely is a bad idea, so
I'd advice you to use time based artificial bucketing to limit the maximum
size of your partitions to be as close as possible to the recommendations.

Also 120+ columns sounds like quite many, is there a way you can separate
in different cfs or maybe use collections? I'd advice to do some
benchmarking here: http://mrcalonso.com/benchmarking-cassandra-models/.
This post is a bit outdated as nowadays you can use cassandra-stress with
your own models, but the idea is the same.

About compactions I'd use DTCS or LCS, but given that you will have a big
amount of tombstones due to TTLs I'd never go with STCS.

Hope it helps!

Carlos Alonso | Software Engineer | @calonso <https://twitter.com/calonso>

On 30 October 2015 at 10:55, <chandrasekar.krc@wipro.com> wrote:

> Hi,
>
>
>
> Could you please suggest if Narrow partition is a  good choice for the
> below use case.
>
>
>
> 1)      Write heavy event log table with 50m inserts per day with a peak
> load of 20K transaction per sec. There aren’t any updates/deletes to
> records inserted. Records are inserted with a TTL of 60 days (retention
> period)
>
> 2)      The table has a single primary key which is a sequence number (27
> digits) generated by source application
>
> 3)      There are only two access patterns used – one by using the
> sequence number & the other using sequence number + event date (range scans
> also possible)
>
> 4)      My target data model in Cassandra is partitioned with sequence
> number as the primary key + event date as clustering columns to enable
> range scans on date.
>
> 5)      The Table has close to 120+ columns and the average row size
> comes close to 32K bytes
>
> 6)      Reads are very very less and account to <5% while inserts can be
> close to 95%.
>
> 7)      From a functional standpoint, I do not see any other columns that
> can be part of primary key to keep the partition reasonable (<100MB)
>
>
>
> Questions:
>
> 1)      Is Narrow partition an ideal choice for the above use case.
>
> 2)      Is artificial bucketing an alternate choice to make the partition
> reasonable
>
> 3)      We are using varint as the data type for sequence number which is
> 27 digits long. Is DECIMAL data type ?
>
> 4)      Any suggestions on performance impacts during compaction ?
>
>
>
> Regards, Chandra Sekar KR
>
>
> The information contained in this electronic message and any attachments
> to this message are intended for the exclusive use of the addressee(s) and
> may contain proprietary, confidential or privileged information. If you are
> not the intended recipient, you should not disseminate, distribute or copy
> this e-mail. Please notify the sender immediately and destroy all copies of
> this message and any attachments. WARNING: Computer viruses can be
> transmitted via email. The recipient should check this email and any
> attachments for the presence of viruses. The company accepts no liability
> for any damage caused by any virus transmitted by this email.
> www.wipro.com
>

Mime
View raw message