cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Subelsky <m...@subelsky.com>
Subject Re: munin is great for monitoring Cassandra
Date Thu, 17 Jun 2010 20:58:22 GMT
T

On Thu, Jun 17, 2010 at 4:53 PM, Peter Schuller
<peter.schuller@infidyne.com> wrote:
>> drive on an EBS volume).  Does that pattern look bad or is it
>> relatively normal? If it's not normal, any advice for what I should
>> change to get better performance?
>
> Assuming you're running with the default cassandra VM options or
> similar (mainly, the CMS GC), the graph looks fairly normal to me. You
> would tend to expect frequent small variations that are the result of
> young-generation GC:s (ParNew:s), with some less frequent larger dips
> in memory usage associated with CMS completing a concurrent mark/sweep
> phase, typically triggered by used heap size approaching the current
> heap size.
>
> Normally the dips are somewhat indicative of the size of the actual
> live set that your application is using; I'm not sure what's going on
> in the middle of the graph with the dip being significantly higher.
> Did you have very significant write activity during that period (but
> it seems excessive even so).
>
> --
> / Peter Schuller

Thanks for looking at this guys.  Peter, I added a columnfamily around
that time which caused me to restart each node.  One of my nodes had a
hard time coming back to life because its small / partition was 100%
full due to two large core dump files - which I'm looking into how to
prevent (or at least change where they get dumped).  I'm guessing the
other two nodes experienced heavier loads during that time.

-Mike



-- 
Mike Subelsky
oib.com // ignitebaltimore.com // subelsky.com
@subelsky // (410) 929-4022

Mime
View raw message