cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joost van de Wijgerd <jwijg...@gmail.com>
Subject Setting the Memtable allocator on a per CF basis
Date Mon, 09 Jul 2012 13:47:38 GMT
Hello Cassandra Devs,

We are currently trying to optimize our Cassandra system with
different workloads. One of our workload is update heavy (very).

Currently we are running with a patch that allows the Live Ratio to go
below 1.0 (lower bound set to 0.1 now) which gives us a
bit better performance in terms of flushes on this particular CF. We
then experienced unexpected memory issues which on further
inspection seems to be related to the SlabAllocator. What happens is
that we allocate a Region of 1MB every couple of seconds (the columns
we write in this CF contain serialized session data, can be 100K
each), so overwrites are actually done into another Region and these
regions are only freed (most of the time) when the Memtable is
flushed. We actually added some debug logs and to write about 300MB to
disk we created roughly 3000 regions. (3GB of data, some of them might
be collected before the flush but probably not much)

It would really great if we could use the native allocator only for
this CF. Since the SlabAllocator gives us very good results on our
other
CFs. (we tried running on a patched version with the HeapAllocator set
but went OOM almost immediately)

I have found this issue in which Jonathan mentions he is ok with
adding a configuration option:

https://issues.apache.org/jira/browse/CASSANDRA-3073

Unfortunately it seems the issue was closed and nothing was implemented.

Would you guys consider adding this option to a future release?
SlabAllocator should be the default but in the CF properties the
HeapAllocator
can be set.

If you want I can try to create a patch myself and submit it to you?

Kind Regards

Joost

-- 
Joost van de Wijgerd
Visseringstraat 21B
1051KH Amsterdam
+31624111401
joost.van.de.wijgerd@Skype
http://www.linkedin.com/in/jwijgerd

Mime
View raw message