incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Georgiev <yngw...@bk.ru>
Subject Re: 0.7.2 slow memtables flushing
Date Sat, 19 Feb 2011 16:32:55 GMT
On 19.2.2011 г. 16:43 ч., Jonathan Ellis wrote:
> Flush code didn't change between 0.7.0 and 0.7.2. There must be some
> other variable here. Memory pressure maybe?

Cloud you please elaborate on that one ?
The conditions are exactly the same for the test with 0.7.0 and 0.7.2.
By the way, 0.7.1 tests are similar to 0.7.2, while 0.7 early betas to 
0.7.0 is fine.

Ivan

On 19.2.2011 г. 16:43 ч., Jonathan Ellis wrote:
> Flush code didn't change between 0.7.0 and 0.7.2. There must be some
> other variable here. Memory pressure maybe?
>
> On Sat, Feb 19, 2011 at 8:16 AM, Ivan Georgiev<yngwiie@bk.ru>  wrote:
>> Hi,
>>
>> I am testing 0.7.2 on a Windows 2003 x64 system(one node) and I am having
>> the following problem.
>> My insertion speed is relatively slow, so the memtables do not get full and
>> the actual flushing is triggered by memtable_flush_after_mins, this happens
>> on the hour mark. My problem with 0.7.2 is that when that happens, the cpu
>> spikes to 25% overall usage (x4 Xeon) and the operation takes anywhere from
>> 2 to 4 minutes, leaving the node not responding during that time. This has
>> forced to me to increase the rpc timeout option to beyond what i feel
>> comfortable with.
>> I have run multiple tests with 0.7.0 and 0.7.2 with the same dataset and the
>> results are consistent. During the same operation 0.7.0 takes about 10
>> seconds to complete vs. 2 to 4 minutes for 0.7.2. I am attaching a log with
>> the timestamps from one such flushing of 0.7.2. Please let me know if there
>> is anything i can do to speed up and get results similar to 0.7.0.
>>
>> Regards:
>> Ivan
>>
>> This is the log of the operations which took most time during the flush
>> operation. Using 0.7.0, with the same number of operations the flushing
>> takes less than 10 seconds.
>>
>> INFO 01:36:44,906 Enqueuing flush of
>> Memtable-ArchiveFiles.6f776e65724944@1225856921(1619949 bytes, 34467
>> operations)
>> INFO 01:37:47,375 Completed flushing
>> C:\Storage\data\Storage\ArchiveFiles.68617368-f-3-Data.db (5549187 bytes)
>> INFO 01:37:47,375 Writing Memtable-ArchiveFiles.6e616d65@978152661(1619949
>> bytes, 34467 operations)
>> INFO 01:37:47,375 Enqueuing flush of
>> Memtable-ArchiveFiles.706172656e74466f6c6465724944@2097700961(1619949 bytes,
>> 34467 operations)
>> INFO 01:38:51,343 Completed flushing
>> C:\Storage\data\Storage\ArchiveFiles.6e616d65-f-3-Data.db (3820265 bytes)
>> INFO 01:38:51,343 Writing
>> Memtable-ArchiveFiles.6f776e65724944@1225856921(1619949 bytes, 34467
>> operations)
>>
>
>


Mime
View raw message