Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 30860 invoked from network); 2 Jun 2010 19:09:49 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 2 Jun 2010 19:09:49 -0000 Received: (qmail 24304 invoked by uid 500); 2 Jun 2010 19:09:48 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 24268 invoked by uid 500); 2 Jun 2010 19:09:48 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 24260 invoked by uid 99); 2 Jun 2010 19:09:48 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Jun 2010 19:09:48 +0000 X-ASF-Spam-Status: No, hits=0.7 required=10.0 tests=RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [74.125.78.25] (HELO ey-out-2122.google.com) (74.125.78.25) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Jun 2010 19:09:40 +0000 Received: by ey-out-2122.google.com with SMTP id 22so10427eye.25 for ; Wed, 02 Jun 2010 12:09:20 -0700 (PDT) MIME-Version: 1.0 Received: by 10.213.31.203 with SMTP id z11mr5277780ebc.86.1275505760047; Wed, 02 Jun 2010 12:09:20 -0700 (PDT) Received: by 10.213.9.129 with HTTP; Wed, 2 Jun 2010 12:09:20 -0700 (PDT) In-Reply-To: <2C1B43AF-880F-4288-BA8A-FA8E33F8DF10@gmail.com> References: <2C1B43AF-880F-4288-BA8A-FA8E33F8DF10@gmail.com> Date: Wed, 2 Jun 2010 21:09:20 +0200 Message-ID: Subject: Re: Continuously increasing RAM usage From: Torsten Curdt To: user@cassandra.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org We've also seen something like this. Will soon investigate and try again with 0.6.2 On Wed, Jun 2, 2010 at 20:27, Paul Brown wrote: > > FWIW, I'm seeing similar issues on a cluster. =A0Three nodes, Cassandra 0= .6.1, SUN JDK 1.6.0_b20. =A0I will try to get some heap dumps to see what's= building up. > > I've seen this sort of issue in systems that make heavy use of java.util.= concurrent queues/executors, e.g.: > > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=3D6236036 > > That bug is long fixed, but it is an instance of how it can be harder to = do nothing than something. > > -- Paul > > > On May 26, 2010, at 11:32 PM, James Golick wrote: > >> We're seeing RAM usage continually climb until eventually, cassandra bec= omes unresponsive. >> >> The JVM isn't OOM'ing. It has only committed 14/24GB of memory. So, I am= assuming that the memory usage is related to mmap'd IO. Fair assumption? >> >> I tried setting the IO mode to standard, but it seemed to be a little sl= ower and couldn't get the machine to come back online with adequate read pe= rformance, so I set it back. I'll have to write a solid cache warming scrip= t if I'm going to try that again. >> >> Any other ideas for what might be causing the issue? Is there something = I should monitor or look at next time it happens? >> >> Thanks > >