incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: MemtableThroughput test in ColumnFamily.apply
Date Mon, 27 Feb 2012 20:00:00 GMT
That sounds odd because it is checked after each row is added to the memtable. 

What are you seeing logged when the memtable flushes ? It will say how many ops and how many
(tracked) bytes. 

Cheers

-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 26/02/2012, at 7:48 AM, Thomas Richter wrote:

> Hi,
> 
> I agree, but the problem in our case is that we have rather small
> memtables (5MB) and during hinted handoff there are several dozen or
> even hundreds of MBs inserted without flushing the tables. And in that
> case it makes a difference.
> 
> Best,
> 
> Thomas
> 
> On 02/25/2012 07:06 PM, Jonathan Ellis wrote:
>> Makes sense to me, although I don't see it making a material
>> difference when there are 1000 mutations in a memtable vs 1001.
>> 
>> On Sat, Feb 25, 2012 at 11:23 AM, Thomas Richter <tho@tricnet.de> wrote:
>>> Hi,
>>> 
>>> while hunting down some memory consumption issues in 0.7.10 I realized
>>> that MemtableThroughput condition is tested before writing the new data.
>>> As this causes memtables to grow larger than expected I changed
>>> 
>>> Memtable apply(DecoratedKey key, ColumnFamily columnFamily)
>>>   {
>>>       long start = System.nanoTime();
>>> 
>>>       boolean flushRequested = memtable.isThresholdViolated();
>>>       memtable.put(key, columnFamily);
>>>       ColumnFamily cachedRow = getRawCachedRow(key);
>>>       if (cachedRow != null)
>>>           cachedRow.addAll(columnFamily);
>>>       writeStats.addNano(System.nanoTime() - start);
>>> 
>>>       return flushRequested ? memtable : null;
>>>   }
>>> 
>>> to
>>> 
>>> Memtable apply(DecoratedKey key, ColumnFamily columnFamily)
>>>   {
>>>       long start = System.nanoTime();
>>> 
>>> 
>>>       memtable.put(key, columnFamily);
>>>       ColumnFamily cachedRow = getRawCachedRow(key);
>>>       if (cachedRow != null)
>>>           cachedRow.addAll(columnFamily);
>>>       writeStats.addNano(System.nanoTime() - start);
>>>       boolean flushRequested = memtable.isThresholdViolated();
>>>       return flushRequested ? memtable : null;
>>>   }
>>> 
>>> Are there any objections to this change? So far it works for me.
>>> 
>>> Best,
>>> 
>>> Thomas
>> 
>> 
>> 


Mime
View raw message