cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: Setting the Memtable allocator on a per CF basis
Date Tue, 10 Jul 2012 09:18:02 GMT
> Would you guys consider adding this option to a future release?
All improvements are considered :) Please create a ticket on
https://issues.apache.org/jira/browse/CASSANDRA and reference CASSANDRA-3073

> If you want I can try to create a patch myself and submit it to you?
Sounds like a plan.

Thanks
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 10/07/2012, at 1:47 AM, Joost van de Wijgerd wrote:

> Hello Cassandra Devs,
> 
> We are currently trying to optimize our Cassandra system with
> different workloads. One of our workload is update heavy (very).
> 
> Currently we are running with a patch that allows the Live Ratio to go
> below 1.0 (lower bound set to 0.1 now) which gives us a
> bit better performance in terms of flushes on this particular CF. We
> then experienced unexpected memory issues which on further
> inspection seems to be related to the SlabAllocator. What happens is
> that we allocate a Region of 1MB every couple of seconds (the columns
> we write in this CF contain serialized session data, can be 100K
> each), so overwrites are actually done into another Region and these
> regions are only freed (most of the time) when the Memtable is
> flushed. We actually added some debug logs and to write about 300MB to
> disk we created roughly 3000 regions. (3GB of data, some of them might
> be collected before the flush but probably not much)
> 
> It would really great if we could use the native allocator only for
> this CF. Since the SlabAllocator gives us very good results on our
> other
> CFs. (we tried running on a patched version with the HeapAllocator set
> but went OOM almost immediately)
> 
> I have found this issue in which Jonathan mentions he is ok with
> adding a configuration option:
> 
> https://issues.apache.org/jira/browse/CASSANDRA-3073
> 
> Unfortunately it seems the issue was closed and nothing was implemented.
> 
> Would you guys consider adding this option to a future release?
> SlabAllocator should be the default but in the CF properties the
> HeapAllocator
> can be set.
> 
> If you want I can try to create a patch myself and submit it to you?
> 
> Kind Regards
> 
> Joost
> 
> -- 
> Joost van de Wijgerd
> Visseringstraat 21B
> 1051KH Amsterdam
> +31624111401
> joost.van.de.wijgerd@Skype
> http://www.linkedin.com/in/jwijgerd


Mime
View raw message