incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boris Yen <yulin...@gmail.com>
Subject Re: MeteredFlusher in system.log entries
Date Sun, 08 Jul 2012 03:14:58 GMT
I am not sure, but I think there should be only 6 memtables (max) based on
the example. 1 is active, 4 are in the queue, 1 is being flushed.

Is this correct?

On Wed, Jun 6, 2012 at 9:08 PM, rohit bhatia <rohit2412@gmail.com> wrote:

> Also, Could someone please explain how the factor of 7 comes in the
> picture in this sentence
>
> "For example if memtable_total_space_in_mb is 100MB, and
> memtable_flush_writers is the default 1 (with one data directory), and
> memtable_flush_queue_size is the default 4, and a Column Family has no
> secondary indexes. The CF will not be allowed to get above one seventh
> of 100MB or 14MB, as if the CF filled the flush pipeline with 7
> memtables of this size it would take 98MB. "
>
> On Wed, Jun 6, 2012 at 6:22 PM, rohit bhatia <rohit2412@gmail.com> wrote:
> > Hi..
> >
> > the link http://thelastpickle.com/2011/05/04/How-are-Memtables-measured/
> > mentions that "From version 0.7 onwards the worse case scenario is up
> > to CF Count + Secondary Index Count + memtable_flush_queue_size
> > (defaults to 4) + memtable_flush_writers (defaults to 1 per data
> > directory) memtables in memory the JVM at once.".
> >
> > So it implies that for flushing, Cassandra copies the memtables content.
> > So does this imply that writes to column families are not stopped even
> > when it is being flushed?
> >
> > Thanks
> > Rohit
> >
> > On Wed, Jun 6, 2012 at 9:42 AM, rohit bhatia <rohit2412@gmail.com>
> wrote:
> >> Hi Aaron
> >>
> >> Thanks for the link, I have gone through it. But this doesn't justify
> >> nodes of exactly same config/specs differing in their flushing
> >> frequency.
> >> The traffic on all node is same as we are using RandomPartitioner
> >>
> >> Thanks
> >> Rohit
> >>
> >> On Wed, Jun 6, 2012 at 12:24 AM, aaron morton <aaron@thelastpickle.com>
> wrote:
> >>> See the section on memtable_total_space_in_mb here
> >>>  http://thelastpickle.com/2011/05/04/How-are-Memtables-measured/
> >>>
> >>> Cheers
> >>> -----------------
> >>> Aaron Morton
> >>> Freelance Developer
> >>> @aaronmorton
> >>> http://www.thelastpickle.com
> >>>
> >>> On 6/06/2012, at 2:27 AM, rohit bhatia wrote:
> >>>
> >>> I am trying to understand the variance in flushes frequency in a 8
> >>> node Cassandra cluster.
> >>> All the flushes are of the same type and initiated by
> MeteredFlusher.java =>
> >>>
> >>> "INFO [OptionalTasks:1] 2012-06-05 06:32:05,873 MeteredFlusher.java
> >>> (line 62) flushing high-traffic column family CFS(Keyspace='Stats',
> >>> ColumnFamily='Minutewise_Channel_Stats') (estimated 501695882 bytes)"
> >>> [taken from system.log]
> >>>
> >>> Number of flushes for 1 column family vary from 6 flushes per day to
> >>> 24 flushes per day among nodes of same configuration and same
> >>> hardware.
> >>> Could you please throw light on the what conditions does
> >>> MeteredFlusher use to trigger memtable flushes.
> >>> Also how accurate is the estimated size in the above logfile entry.
> >>>
> >>> Regards
> >>> Rohit Bhatia
> >>> Software Engineer, Media.net
> >>>
> >>>
>

Mime
View raw message