Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 290B19017 for ; Mon, 12 Dec 2011 15:33:37 +0000 (UTC) Received: (qmail 16842 invoked by uid 500); 12 Dec 2011 15:33:34 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 16813 invoked by uid 500); 12 Dec 2011 15:33:34 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 16805 invoked by uid 99); 12 Dec 2011 15:33:34 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Dec 2011 15:33:34 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of watcherfr@gmail.com designates 74.125.82.44 as permitted sender) Received: from [74.125.82.44] (HELO mail-ww0-f44.google.com) (74.125.82.44) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Dec 2011 15:33:27 +0000 Received: by mail-ww0-f44.google.com with SMTP id dr13so9514455wgb.25 for ; Mon, 12 Dec 2011 07:33:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=qR3pokfMThgJDjcWrQK1+0cWhEh9MadF3OeXCEJygSs=; b=EQdOSyx8dajQdcSJBlhq94Ia09jmM1MWAPzPUP2QcJ9qa6ipDHImptPR75KiJqlz4r Jcrv9CFGOaHiGzxarbILrijpdJe0x3ZABg0z/lGYOlYh3wDkzDRgygJQNOr2HGNKXf1F AaA94f84OlI3bXMqOFF8qgM2xb9RtkdHdH7eM= MIME-Version: 1.0 Received: by 10.227.208.129 with SMTP id gc1mr17694686wbb.4.1323703986588; Mon, 12 Dec 2011 07:33:06 -0800 (PST) Received: by 10.180.83.70 with HTTP; Mon, 12 Dec 2011 07:33:06 -0800 (PST) In-Reply-To: <4EE5F94C.5090208@futureful.com> References: <4EE5F94C.5090208@futureful.com> Date: Mon, 12 Dec 2011 16:33:06 +0100 Message-ID: Subject: Re: a query that's killing cassandra From: Philippe To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0015174a050ce88c1e04b3e6d9a6 --0015174a050ce88c1e04b3e6d9a6 Content-Type: text/plain; charset=ISO-8859-1 You've got at least one row over 1GB, compacted ! Have you checked whether you are running out of heap ? 2011/12/12 Wojtek Kulik > Hello everyone! > > I have a strange problem with Cassandra (v1.0.5, one node, 8GB, 2xCPU): a > query asking for each key from a certain (super) CF results in timeout and > almost dead cassandra after that (it's somewhat alive, but does not return > any data - has to be restarted). > > CF details: > Column Family: ************ > SSTable count: 11 > Space used (live): 4254606435 > Space used (total): 4254606435 > Number of Keys (estimate): 1408 > Memtable Columns Count: 26923 > Memtable Data Size: 25420941 > Memtable Switch Count: 16 > Read Count: 0 > Read Latency: NaN ms. > Write Count: 39649 > Write Latency: 0.032 ms. > Pending Tasks: 0 > Bloom Filter False Postives: 0 > Bloom Filter False Ratio: 0.00000 > Bloom Filter Space Used: 30416 > Key cache capacity: 24 > Key cache size: 24 > Key cache hit rate: NaN > Row cache: disabled > Compacted row minimum size: 125 > Compacted row maximum size: 1155149911 > Compacted row mean size: 38855255 > > Nothing suspicious in the logs. The problem is fully replicable. > I've spent some time searching for similar issues - haven't found any. > > Are there any debug options I could turn on to find out more? Other > suggestions/thoughts? > > Thanks in advance, > Wojtek > --0015174a050ce88c1e04b3e6d9a6 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable You've got at least one row over 1GB, compacted !
Have you checked = whether you are running out of heap ?

201= 1/12/12 Wojtek Kulik <wojtek.kulik@futureful.com>
Hello everyone!

I have a strange problem with Cassandra (v1.0.5, one node, 8GB, 2xCPU): a q= uery asking for each key from a certain (super) CF results in timeout and a= lmost dead cassandra after that (it's somewhat alive, but does not retu= rn any data - has to be restarted).

CF details:
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Column Family: ************
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0SSTable count: 11
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Space used (live): 4254606435
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Space used (total): 4254606435
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Number of Keys (estimate): 1408
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Memtable Columns Count: 26923
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Memtable Data Size: 25420941
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Memtable Switch Count: 16
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Read Count: 0
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Read Latency: NaN ms.
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Write Count: 39649
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Write Latency: 0.032 ms.
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Pending Tasks: 0
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Bloom Filter False Postives: 0
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Bloom Filter False Ratio: 0.00000
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Bloom Filter Space Used: 30416
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Key cache capacity: 24
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Key cache size: 24
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Key cache hit rate: NaN
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Row cache: disabled
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Compacted row minimum size: 125
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Compacted row maximum size: 1155149911
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Compacted row mean size: 38855255

Nothing suspicious in the logs. The problem is fully replicable.
I've spent some time searching for similar issues - haven't found a= ny.

Are there any debug options I could turn on to find out more? Other suggest= ions/thoughts?

Thanks in advance,
Wojtek

--0015174a050ce88c1e04b3e6d9a6--