incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anand Somani <meatfor...@gmail.com>
Subject Re: Commit log fills up in less than a minute
Date Sun, 21 Aug 2011 16:09:28 GMT
So no it did not fill in a minute, but ton's of header files were written in
a minute (is that normal, I assume these are marker files which get written
when memtables are flushed. The actual data files have been around for the
last 24 hours?
Somehow this all seems connected to "reintroduce node" exercise I went thru
on Saturday. Somehow can this get worse with cleanup, hinted hand off,
etc....

When does the actual commit-data file get deleted.

The flush interval on all my memtables is 60 minutes

Thanks

On Sun, Aug 21, 2011 at 8:43 AM, Anand Somani <meatforums@gmail.com> wrote:

> Hi,
>
> 7.4, 3 node cluster, RF=3
>
> Load has not changed much, on 2 of the 3 nodes the commit log filled up in
> less than a minute (did not give a chance to recover). Now have been running
> this cluster for abt 2-3 months without any problem. At this point I do not
> see any unusual load (continue to investigate). The commit log has never
> taken more than 20% before this!!!
>
> The only changes was a re-intro of a node couple days ago with a bunch of
> cleanup on each node. Has anybody seen anything like this?
>
> I am starting the entire cluster again, anyway hints and things I should
> watch for (besides commit -log). I have backed up the commit logs and logs,
> so any tips on how to do a analysis on this would help.
>
> Thanks
> Anand
>

Mime
View raw message