Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4D7981028B for ; Thu, 27 Jun 2013 19:45:48 +0000 (UTC) Received: (qmail 48980 invoked by uid 500); 27 Jun 2013 19:45:45 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 48953 invoked by uid 500); 27 Jun 2013 19:45:45 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 48945 invoked by uid 99); 27 Jun 2013 19:45:45 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Jun 2013 19:45:45 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of kohlisankalp@gmail.com designates 209.85.216.47 as permitted sender) Received: from [209.85.216.47] (HELO mail-qa0-f47.google.com) (209.85.216.47) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Jun 2013 19:45:40 +0000 Received: by mail-qa0-f47.google.com with SMTP id i13so56123qae.6 for ; Thu, 27 Jun 2013 12:45:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=Jjt2qssel9vYZbbBOLirtpp3/Nm8PJiCLDgV6LtipF8=; b=O4Z/SvRooIkoAvL1XicqZvoS0luqYj7cZK2a+6+yAdzyVTIvaIe2cbPwgD8tNJ8OKz AgtuCqmc6OywRx3i9ggO0fWYH4e73N2xIYIIgB0ZwM/N6fmHPqz1nFLCpIIglqOxLmtr nRdc+sXdE9088LByI4F9gGBwsg70FVDW6AesZglcfx5Vo4i9LarV+R4lgpkQN+rJ7GVY PpmBgFMge0+RM3FJov5GlWNItogWlmwqJxEgUQqfmbeONq4pHQKhYBNDTUm13jsM9JBq Np/sBH+hbBYp8gQfF6WjMnN30eMcsZvWLthTwTYLkAL66bRoh8oDpJpb3HNg7LdT80Xj p+TQ== X-Received: by 10.229.142.202 with SMTP id r10mr2950675qcu.81.1372362319418; Thu, 27 Jun 2013 12:45:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.49.64.7 with HTTP; Thu, 27 Jun 2013 12:44:39 -0700 (PDT) In-Reply-To: <1372358454.85687.YahooMailNeo@web142804.mail.bf1.yahoo.com> References: <1372358454.85687.YahooMailNeo@web142804.mail.bf1.yahoo.com> From: sankalp kohli Date: Thu, 27 Jun 2013 15:44:39 -0400 Message-ID: Subject: Re: High-read latency for non-existing rows with LCS and 1.2.5 To: user@cassandra.apache.org, Bao Le Content-Type: multipart/alternative; boundary=90e6ba10b0018d7e5c04e02800ad X-Virus-Checked: Checked by ClamAV on apache.org --90e6ba10b0018d7e5c04e02800ad Content-Type: text/plain; charset=ISO-8859-1 Try doing request tracing. http://www.datastax.com/dev/blog/tracing-in-cassandra-1-2 On Thu, Jun 27, 2013 at 2:40 PM, Bao Le wrote: > Hi, > > We are using Leveled Compaction with Cassandra 1.2.5. Our sstable size > is 100M. On each node, > we have anywhere from 700+ to 800+ sstables (for all levels). The > bloom_filter_fp_chance is set at 0.000744. > > For read requests that ask for existing row records, the latency is > great, mostly under 20 milliseconds with key cache and row cache set. For > read requests that ask for non-existing row records (not because of > deletes, but rather, have never been in the system to start with), the > latency is running into hundreds of milliseconds and sometimes seconds. > > Just wonder if anyone has come across this before and has some pointers > on how to reduce the latency in this case. > > Thanks > Bao > > > > --90e6ba10b0018d7e5c04e02800ad Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
=

On Thu, Jun 27, 2013 at 2:40 PM, Bao Le <lvqb@yaho= o.com> wrote:
Hi,

=A0=A0=A0 We are using Leveled Compaction with Cassandr= a 1.2.5. Our sstable size is 100M. On each node,
we have anywhere from 7= 00+ to 800+ sstables (for all levels). The bloom_filter_fp_chance is set at= 0.000744.

=A0 For read requests that ask for existing row records, the latency is= great, mostly under 20 milliseconds with key cache and row cache set. For = read requests that ask for non-existing row records (not because of deletes= , but rather, have never been in the system to start with), the latency is = running into hundreds of milliseconds and sometimes seconds.

=A0 Just wonder if anyone has come across this before and has some poin= ters on how to reduce the latency in this case.

Thanks
Bao



<= /span>

--90e6ba10b0018d7e5c04e02800ad--