incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Idrén, Johan <Johan.Id...@dice.se>
Subject RE: memtable mem usage off by 10?
Date Wed, 04 Jun 2014 10:04:01 GMT
I'm not measuring memtable size by looking at the sstables on disk, no. I'm looking through
the JMX data. So I would believe (or hope) that I'm getting relevant data.


If I have a heap of 10GB and set the memtable usage to 20GB, I would expect to hit other problems,
but I'm not seeing memory usage over 10GB for the heap, and the machine (which has ~30gb of
memory) is showing ~10GB free, with ~12GB used by cassandra, the rest in caches.


Reading 8k rows/s, writing 2k rows/s on a 3 node cluster. So it's not idling.


BR

Johan


________________________________
From: Benedict Elliott Smith <belliottsmith@datastax.com>
Sent: Wednesday, June 4, 2014 11:56 AM
To: user@cassandra.apache.org
Subject: Re: memtable mem usage off by 10?

If you are storing small values in your columns, the object overhead is very substantial.
So what is 400Mb on disk may well be 4Gb in memtables, so if you are measuring the memtable
size by the resulting sstable size, you are not getting an accurate picture. This overhead
has been reduced by about 90% in the upcoming 2.1 release, through tickets 6271<https://issues.apache.org/jira/browse/CASSANDRA-6271>,
6689<https://issues.apache.org/jira/browse/CASSANDRA-6689> and 6694<https://issues.apache.org/jira/browse/CASSANDRA-6694>.


On 4 June 2014 10:49, Idrén, Johan <Johan.Idren@dice.se<mailto:Johan.Idren@dice.se>>
wrote:

Hi,


I'm seeing some strange behavior of the memtables, both in 1.2.13 and 2.0.7, basically it
looks like it's using 10x less memory than it should based on the documentation and options.


10GB heap for both clusters.

1.2.x should use 1/3 of the heap for memtables, but it uses max ~300mb before flushing


2.0.7, same but 1/4 and ~250mb


In the 2.0.7 cluster I set the memtable_total_space_in_mb to 4096, which then allowed cassandra
to use up to ~400mb for memtables...


I'm now running with 20480 for memtable_total_space_in_mb and cassandra is using ~2GB for
memtables.


Soo, off by 10 somewhere? Has anyone else seen this? Can't find a JIRA for any bug connected
to this.

java 1.7.0_55, JNA 4.1.0 (for the 2.0 cluster)


BR

Johan


Mime
View raw message