cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rohit bhatia <rohit2...@gmail.com>
Subject Re: MeteredFlusher in system.log entries
Date Sun, 08 Jul 2012 06:09:22 GMT
@boris https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/MeteredFlusher.java#L51

On Sun, Jul 8, 2012 at 8:44 AM, Boris Yen <yulinyen@gmail.com> wrote:
> I am not sure, but I think there should be only 6 memtables (max) based on
> the example. 1 is active, 4 are in the queue, 1 is being flushed.
>
> Is this correct?
>
>
> On Wed, Jun 6, 2012 at 9:08 PM, rohit bhatia <rohit2412@gmail.com> wrote:
>>
>> Also, Could someone please explain how the factor of 7 comes in the
>> picture in this sentence
>>
>> "For example if memtable_total_space_in_mb is 100MB, and
>> memtable_flush_writers is the default 1 (with one data directory), and
>> memtable_flush_queue_size is the default 4, and a Column Family has no
>> secondary indexes. The CF will not be allowed to get above one seventh
>> of 100MB or 14MB, as if the CF filled the flush pipeline with 7
>> memtables of this size it would take 98MB. "
>>
>> On Wed, Jun 6, 2012 at 6:22 PM, rohit bhatia <rohit2412@gmail.com> wrote:
>> > Hi..
>> >
>> > the link http://thelastpickle.com/2011/05/04/How-are-Memtables-measured/
>> > mentions that "From version 0.7 onwards the worse case scenario is up
>> > to CF Count + Secondary Index Count + memtable_flush_queue_size
>> > (defaults to 4) + memtable_flush_writers (defaults to 1 per data
>> > directory) memtables in memory the JVM at once.".
>> >
>> > So it implies that for flushing, Cassandra copies the memtables content.
>> > So does this imply that writes to column families are not stopped even
>> > when it is being flushed?
>> >
>> > Thanks
>> > Rohit
>> >
>> > On Wed, Jun 6, 2012 at 9:42 AM, rohit bhatia <rohit2412@gmail.com>
>> > wrote:
>> >> Hi Aaron
>> >>
>> >> Thanks for the link, I have gone through it. But this doesn't justify
>> >> nodes of exactly same config/specs differing in their flushing
>> >> frequency.
>> >> The traffic on all node is same as we are using RandomPartitioner
>> >>
>> >> Thanks
>> >> Rohit
>> >>
>> >> On Wed, Jun 6, 2012 at 12:24 AM, aaron morton <aaron@thelastpickle.com>
>> >> wrote:
>> >>> See the section on memtable_total_space_in_mb here
>> >>>  http://thelastpickle.com/2011/05/04/How-are-Memtables-measured/
>> >>>
>> >>> Cheers
>> >>> -----------------
>> >>> Aaron Morton
>> >>> Freelance Developer
>> >>> @aaronmorton
>> >>> http://www.thelastpickle.com
>> >>>
>> >>> On 6/06/2012, at 2:27 AM, rohit bhatia wrote:
>> >>>
>> >>> I am trying to understand the variance in flushes frequency in a 8
>> >>> node Cassandra cluster.
>> >>> All the flushes are of the same type and initiated by
>> >>> MeteredFlusher.java =>
>> >>>
>> >>> "INFO [OptionalTasks:1] 2012-06-05 06:32:05,873 MeteredFlusher.java
>> >>> (line 62) flushing high-traffic column family CFS(Keyspace='Stats',
>> >>> ColumnFamily='Minutewise_Channel_Stats') (estimated 501695882 bytes)"
>> >>> [taken from system.log]
>> >>>
>> >>> Number of flushes for 1 column family vary from 6 flushes per day to
>> >>> 24 flushes per day among nodes of same configuration and same
>> >>> hardware.
>> >>> Could you please throw light on the what conditions does
>> >>> MeteredFlusher use to trigger memtable flushes.
>> >>> Also how accurate is the estimated size in the above logfile entry.
>> >>>
>> >>> Regards
>> >>> Rohit Bhatia
>> >>> Software Engineer, Media.net
>> >>>
>> >>>
>
>

Mime
View raw message