incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Narendra Sharma <narendra.sha...@gmail.com>
Subject Re: memtable_threshold
Date Mon, 28 Mar 2011 21:52:58 GMT
Following shows how the size of memtable is updated:
currentThroughput.addAndGet(cf.size());

The jconsole/JMX shows this and this doesn't account for the overhead of
holding the data in in-memory data structures.

The size of CF, SuperColumn and Column is calculated as following:

Column Size:
public int size()
    {
        /*
         * Size of a column is =
         *   size of a name (short + length of the string)
         * + 1 byte to indicate if the column has been deleted
         * + 8 bytes for timestamp
         * + 4 bytes which basically indicates the size of the byte array
         * + entire byte array.
        */
        return DBConstants.shortSize_ + name.remaining() +
DBConstants.boolSize_ + DBConstants.tsSize_ + DBConstants.intSize_ +
value.remaining();
    }

SuperColumn Size:
public int size()
    {
        int size = 0;
        for (IColumn subColumn : getSubColumns())
        {
            size += subColumn.serializedSize();
        }
        return size;
    }

ColumnFamily Size:
int size()
    {
        int size = 0;
        for (IColumn column : columns.values())
        {
            size += column.size();
        }
        return size;
    }

Hope this makes it clear.

Thanks,
Naren

On Mon, Mar 28, 2011 at 2:15 PM, ruslan usifov <ruslan.usifov@gmail.com>wrote:

>
>
> 2011/3/29 Narendra Sharma <narendra.sharma@gmail.com>
>
>> This is because the memtable threshold is not correct to the last byte.
>> The threshold basically account for column name, value and timestamp (or the
>> serialized column). It doesn't account for all the in-memory overhead for
>> maintaining the data and references etc.
>>
>>
> Overhead in 2 times???? Hm why so many. Also in JMX (throw jconsole) i
> don't see any overhead, when memtable reach it memory threshold it will be
> reset to very low value (about 300 - 400 KB)
>



-- 
Narendra Sharma
Solution Architect
*http://www.persistentsys.com*
*http://narendrasharma.blogspot.com/*

Mime
View raw message