incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benedict Elliott Smith <belliottsm...@datastax.com>
Subject Re: memtable mem usage off by 10?
Date Wed, 04 Jun 2014 10:33:39 GMT
These measurements tell you the amount of user data stored in the
memtables, not the amount of heap used to store it, so the same applies.


On 4 June 2014 11:04, Idrén, Johan <Johan.Idren@dice.se> wrote:

>  I'm not measuring memtable size by looking at the sstables on disk, no.
> I'm looking through the JMX data. So I would believe (or hope) that I'm
> getting relevant data.
>
>
>  If I have a heap of 10GB and set the memtable usage to 20GB, I would
> expect to hit other problems, but I'm not seeing memory usage over 10GB for
> the heap, and the machine (which has ~30gb of memory) is showing ~10GB
> free, with ~12GB used by cassandra, the rest in caches.
>
>
>  Reading 8k rows/s, writing 2k rows/s on a 3 node cluster. So it's not
> idling.
>
>
>  BR
>
> Johan
>
>
>  ------------------------------
> *From:* Benedict Elliott Smith <belliottsmith@datastax.com>
> *Sent:* Wednesday, June 4, 2014 11:56 AM
> *To:* user@cassandra.apache.org
> *Subject:* Re: memtable mem usage off by 10?
>
>  If you are storing small values in your columns, the object overhead is
> very substantial. So what is 400Mb on disk may well be 4Gb in memtables, so
> if you are measuring the memtable size by the resulting sstable size, you
> are not getting an accurate picture. This overhead has been reduced by
> about 90% in the upcoming 2.1 release, through tickets 6271
> <https://issues.apache.org/jira/browse/CASSANDRA-6271>, 6689
> <https://issues.apache.org/jira/browse/CASSANDRA-6689> and 6694
> <https://issues.apache.org/jira/browse/CASSANDRA-6694>.
>
>
> On 4 June 2014 10:49, Idrén, Johan <Johan.Idren@dice.se> wrote:
>
>>  Hi,
>>
>>
>>  I'm seeing some strange behavior of the memtables, both in 1.2.13 and
>> 2.0.7, basically it looks like it's using 10x less memory than it should
>> based on the documentation and options.
>>
>>
>>  10GB heap for both clusters.
>>
>> 1.2.x should use 1/3 of the heap for memtables, but it uses max ~300mb
>> before flushing
>>
>>
>>  2.0.7, same but 1/4 and ~250mb
>>
>>
>>  In the 2.0.7 cluster I set the memtable_total_space_in_mb to 4096,
>> which then allowed cassandra to use up to ~400mb for memtables...
>>
>>
>>  I'm now running with 20480 for memtable_total_space_in_mb and cassandra
>> is using ~2GB for memtables.
>>
>>
>>  Soo, off by 10 somewhere? Has anyone else seen this? Can't find a JIRA
>> for any bug connected to this.
>>
>> java 1.7.0_55, JNA 4.1.0 (for the 2.0 cluster)
>>
>>
>>  BR
>>
>> Johan
>>
>
>

Mime
View raw message