Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of moyesyside@googlemail.com
 designates 209.85.212.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=xGa+kUPVECXwKSHSXhmAYkEEhbPkqXNdvPu18VLe8Egp1OR/o2/eQggjdXUVOUqn3A
         tIAAwHWvKoF/5u5IghvI5NKW73UqLcEhlZK4f4m1hDXYRH3PC6BrUfojhyWrL/DlDPX7
         s829oH0EUGqmnw4phbUqtUVrMgwuyanWE/arE=
MIME-Version: 1.0
In-Reply-To: <AANLkTikOCDmeU6TAAi9vG-DNhBUfafjQ=esToX3gCjBd@mail.gmail.com>
References: <B3A4A9FD-974D-4E28-9044-EE93F25D4E21@googlemail.com>
	<AANLkTikOCDmeU6TAAi9vG-DNhBUfafjQ=esToX3gCjBd@mail.gmail.com>
Date: Sun, 12 Dec 2010 19:07:27 +1100
Message-ID: <AANLkTim79pgkzBze6unPEhJydc1RW1evg7XkFL2DDefC@mail.gmail.com>
Subject: Re: OutOfMemory on count on cassandra 0.6.8 for large number of
 columns
From: Dave Martin <moyesyside@googlemail.com>
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Thanks Tyler. I was unaware of counters.

The use case for column counts is really from a operational perspective,
to allow a sysadmin to do adhoc checks on columns to see if something
has gone wrong in software outside of cassandra.

I think running a cassandra-cli command such as count, which makes
cassandra fall over is not ideal,
unless we can say for X number of columns cassandra needs at least Y
memory allocation for stability.

Cheers

Dave


On Sun, Dec 12, 2010 at 6:39 PM, Tyler Hobbs <tyler@riptano.com> wrote:
> Cassandra has to deserialize all of the columns in the row for get_count(=
).
> So from Cassandra's perspective, it's almost as much work as getting the
> entire row, it just doesn't have to send everything back over the network=
.
>
> If you're frequently counting 8 million columns (or really, anything
> significant), you need to use counters instead.=A0 If this is a rare
> occurrence, you can do the count in multiple chunks by using a starting a=
nd
> ending column in the SlicePredicate for each chunk, but this requires som=
e
> rough knowledge about the distribution of the column names in the row.
>
> - Tyler