incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vivek Mishra <vivek.mis...@impetus.co.in>
Subject unsubscribe
Date Fri, 23 Sep 2011 05:48:58 GMT


-----Original Message-----
From: Yang [mailto:teddyyyy123@gmail.com]
Sent: Friday, September 23, 2011 11:18 AM
To: user@cassandra.apache.org
Subject: Re: is it possible for light-traffic CF to hold down many commit logs?

thanks for the input.

if that's the case, I think the solution would be to sort the CFs to flush by a more complex
criteria than just size. for example the number of dirty commit logs that contain this CF
should be considered as a score.

Yang

On Thu, Sep 22, 2011 at 10:40 PM, Philippe <watcherfr@gmail.com> wrote:
> It sure looks like what I'm seeing on my cluster where a 100G commit
> lot partition fills up in 12 hours (0.8.x)
>
> Le 23 sept. 2011 03:45, "Yang" <teddyyyy123@gmail.com> a écrit :
>> in 1.0.0 we don't have memtable_throughput for each individual CF ,
>> and instead which memtable/CF to flush is determined by "largest
>> getTotalMemtableLiveSize() ".
>> (MeteredFlusher.java line 81)
>>
>>
>> what would happen in the following case ? : I have only 2 CF, the
>> traffic for one CF is 1000 times that of the second CF, so the
>> high-traffic CF constantly triggers total mem threshold , and every
>> time, the busy CF is flushed.
>>
>> but the light-traffic CF is never flushed ( well, until we have
>> flushed about 1000 times the busy CF), now we are left with many
>> commit logs , each of them containing a few entries for the
>> light-traffic table. we have to keep these commit logs because these
>> entries are not flushed to sstable yet.
>>
>> then there are 2 problems: 1) to persist the few records from the
>> light-traffic CF, you have to keep 1000 times the commit logs
>> necessary, taking up disk space 2) when you do a recover on server
>> restart, you'll have to read through all those commit logs .
>>
>> does the above hypothesis sound right?
>>
>> Thanks
>> Yang
>

________________________________

Hear Impetus’ expert talk about Next Gen Big Data Architectures at Strata Conference, NYC
on Sept 22-23. http://t.co/GD2L0N8g. Listen to recording of Impetus Webinar ‘Utilizing Cloud-
based Services for Enterprise Mobility’ at http://bit.ly/pqV4vj.

Click http://www.impetus.com to know more. Follow us on www.twitter.com/impetuscalling


NOTE: This message may contain information that is confidential, proprietary, privileged or
otherwise protected by law. The message is intended solely for the named addressee. If received
in error, please destroy and notify the sender. Any use of this email is prohibited when received
in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors, virus, interception
or interference.
Mime
View raw message