cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Schuller <peter.schul...@infidyne.com>
Subject Re: Cassandra to store 1 billion small 64KB Blobs
Date Fri, 23 Jul 2010 19:33:18 GMT
> We plan to use cassandra as a data storage on at least 2 nodes with RF=2
> for about 1 billion small files.
> We do have about 48TB discspace behind for each node.
>
> now my question is - is this possible with cassandra - reliable - means
> (every blob is stored on 2 jbods)..
>
> we may grow up to nearly 40TB or more on cassandra "storage" data ...
>
> anyone out did something similar?

Other than what Jonathan Shook mentioned, I'd expect one potential
problem to be the number of sstables. At 40 TB, the larger compactions
are going to take quite some time. How many memtables will be flushed
to disk during the time it takes to perform a ~ 40 TB compaction? That
may or may not be an issue depending on how fast writes will happen,
how large your memtables are (the bigger the better) and what your
reads will look like.

(This relates to another thread where I posted about concurrent
compaction, but right now Cassandra only does a single compaction at a
time.)

-- 
/ Peter Schuller

Mime
View raw message