I've started seeing this issue as well. Running 0.6.2.
One interesting thing I happened upon, I explicitly called the GC via
jconsole and the heap dropped completely fixing the issue. When you
explicitly call System.gc() it does a full sweep. I'm wondering if this
issue is to do with the GC flags used.
-Jake
On Wed, Jun 2, 2010 at 3:09 PM, Torsten Curdt <tcurdt@vafer.org> wrote:
> We've also seen something like this. Will soon investigate and try
> again with 0.6.2
>
> On Wed, Jun 2, 2010 at 20:27, Paul Brown <paulrbrown@gmail.com> wrote:
> >
> > FWIW, I'm seeing similar issues on a cluster. Three nodes, Cassandra
> 0.6.1, SUN JDK 1.6.0_b20. I will try to get some heap dumps to see what's
> building up.
> >
> > I've seen this sort of issue in systems that make heavy use of
> java.util.concurrent queues/executors, e.g.:
> >
> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6236036
> >
> > That bug is long fixed, but it is an instance of how it can be harder to
> do nothing than something.
> >
> > -- Paul
> >
> >
> > On May 26, 2010, at 11:32 PM, James Golick wrote:
> >
> >> We're seeing RAM usage continually climb until eventually, cassandra
> becomes unresponsive.
> >>
> >> The JVM isn't OOM'ing. It has only committed 14/24GB of memory. So, I am
> assuming that the memory usage is related to mmap'd IO. Fair assumption?
> >>
> >> I tried setting the IO mode to standard, but it seemed to be a little
> slower and couldn't get the machine to come back online with adequate read
> performance, so I set it back. I'll have to write a solid cache warming
> script if I'm going to try that again.
> >>
> >> Any other ideas for what might be causing the issue? Is there something
> I should monitor or look at next time it happens?
> >>
> >> Thanks
> >
> >
>
|