Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 20512 invoked from network); 13 Jun 2010 09:08:40 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 13 Jun 2010 09:08:40 -0000 Received: (qmail 76447 invoked by uid 500); 13 Jun 2010 09:08:39 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 76432 invoked by uid 500); 13 Jun 2010 09:08:36 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 76424 invoked by uid 99); 13 Jun 2010 09:08:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Jun 2010 09:08:35 +0000 X-ASF-Spam-Status: No, hits=0.7 required=10.0 tests=RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [74.125.82.172] (HELO mail-wy0-f172.google.com) (74.125.82.172) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Jun 2010 09:08:26 +0000 Received: by wyi11 with SMTP id 11so1910966wyi.31 for ; Sun, 13 Jun 2010 02:08:05 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.169.211 with SMTP id n61mr1140541wel.41.1276420085611; Sun, 13 Jun 2010 02:08:05 -0700 (PDT) Sender: scode@scode.org Received: by 10.216.36.133 with HTTP; Sun, 13 Jun 2010 02:08:05 -0700 (PDT) X-Originating-IP: [213.114.156.79] In-Reply-To: References: Date: Sun, 13 Jun 2010 11:08:05 +0200 X-Google-Sender-Auth: GP3sF1BbmeLYYnWMd61twNFIZrE Message-ID: Subject: Re: GC Storm From: Peter Schuller To: user@cassandra.apache.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org > We've also seen similar problems > > =C2=A0https://issues.apache.org/jira/browse/CASSANDRA-1177 To be clear though; un-*flushed* data is very different from un-*compacted* data and the above seems to be about unflushed data? In my test case there was no problem at all flushing data. But my test was sustained write speeds up to 200 million rows and ~ 200 gb of space, and as the database grew larger the compaction goes goes up (as expected). For that particular workload it would probably have been beneficial to have a configurable concurrency on compactions as well (similar to how it is configurable for sstable flushing) because CPU was the bottleneck in the compaction process (i.e., after stopping inserts and letting compaction complete in the background, the compaction thread was CPU-bound and there was plenty of available I/O capacity). --=20 / Peter Schuller