Return-Path: Delivered-To: apmail-incubator-cassandra-user-archive@minotaur.apache.org Received: (qmail 83467 invoked from network); 5 Dec 2009 04:35:29 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 5 Dec 2009 04:35:29 -0000 Received: (qmail 97347 invoked by uid 500); 5 Dec 2009 04:35:28 -0000 Delivered-To: apmail-incubator-cassandra-user-archive@incubator.apache.org Received: (qmail 97279 invoked by uid 500); 5 Dec 2009 04:35:26 -0000 Mailing-List: contact cassandra-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: cassandra-user@incubator.apache.org Delivered-To: mailing list cassandra-user@incubator.apache.org Received: (qmail 97270 invoked by uid 99); 5 Dec 2009 04:35:26 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Dec 2009 04:35:26 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of tve@rightscale.com designates 209.85.210.174 as permitted sender) Received: from [209.85.210.174] (HELO mail-yx0-f174.google.com) (209.85.210.174) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Dec 2009 04:35:17 +0000 Received: by yxe4 with SMTP id 4so2580552yxe.32 for ; Fri, 04 Dec 2009 20:34:55 -0800 (PST) Received: by 10.150.21.20 with SMTP id 20mr2184104ybu.117.1259987695646; Fri, 04 Dec 2009 20:34:55 -0800 (PST) Received: from ?192.168.0.2? (sb0-cf9a65c9.dsl.impulse.net [207.154.101.201]) by mx.google.com with ESMTPS id 6sm1415438ywc.39.2009.12.04.20.34.53 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 04 Dec 2009 20:34:54 -0800 (PST) Message-ID: <4B19E2EB.6020103@rightscale.com> Date: Fri, 04 Dec 2009 20:34:51 -0800 From: Thorsten von Eicken Organization: RightScale, Inc. User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: cassandra-user@incubator.apache.org Subject: Re: Persistently increasing read latency References: <59DD1BA8FD3C0F4C90771C18F2B5B53A4C850190C6@GVW0432EXB.americas.hpqcorp.net> <59DD1BA8FD3C0F4C90771C18F2B5B53A4C850C20C5@GVW0432EXB.americas.hpqcorp.net> <59DD1BA8FD3C0F4C90771C18F2B5B53A4C850C20F4@GVW0432EXB.americas.hpqcorp.net> <59DD1BA8FD3C0F4C90771C18F2B5B53A4C850C228B@GVW0432EXB.americas.hpqcorp.net> <59DD1BA8FD3C0F4C90771C18F2B5B53A4C850C22D8@GVW0432EXB.americas.hpqcorp.net> <59DD1BA8FD3C0F4C90771C18F2B5B53A4C850C2317@GVW0432EXB.americas.hpqcorp.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org It sounds like the system is limited by bytes/sec not by ops/sec since the bottleneck seems to be disk bandwidth. It might be helpful to compute bytes written per sec at the API level, and then estimate how often each "API byte" is written to disk and read from disk due to compactions. That would support some quick back of the envelope calculations to do a performance smell test... Thorsten Jonathan Ellis wrote: > Thanks for looking into this. Doesn't seem like there's much > low-hanging fruit to make compaction faster but I'll keep that in the > back of my mind. > > -Jonathan > >