Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of gdusbabek@gmail.com designates
 74.125.83.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:reply-to:in-reply-to:references:date:message-id
         :subject:from:to:content-type:content-transfer-encoding;
        b=CUj5ZMBD4E26NUTEJluCevt/nF5IsBIWzv2B4TmRydDSqHMe+MYUXaSlmAl0D0hG5b
         ykEXUsEw5s6rErvOVvdsKX672IhWu6OEKl9ien7fr86A/NlbEtO5+0OdzU7xfxqMsgTh
         EzZICc0LtH0VerbgcMjFCwZQp0StNeHAKA3xI=
MIME-Version: 1.0
Reply-To: gdusbabek@gmail.com
In-Reply-To: <k2g7ca6fc441004180914u4b48f843w292eb403644da57c@mail.gmail.com>
References: <i2y7e99ee5f1004152328x8fb29b12jc18e8e4dd38e995@mail.gmail.com>
	 <p2r7ca6fc441004160138lbed0a77ch35e28f0288b30368@mail.gmail.com>
	 <4BC88D97.7050104@gmail.com>
	 <z2t7ca6fc441004170610j7462e627r1716b3bf22afdb4b@mail.gmail.com>
	 <y2z1e85dd391004170646u19ce4239r73519d8fdd7eb56e@mail.gmail.com>
	 <t2z7ca6fc441004170850w2cb21dfcv2e6ef2dc07415ad@mail.gmail.com>
	 <q2r1c6fcff1004180541n4d375789td69e615d1e6495e4@mail.gmail.com>
	 <k2g7ca6fc441004180914u4b48f843w292eb403644da57c@mail.gmail.com>
Date: Mon, 19 Apr 2010 08:49:27 -0500
Message-ID: <r2x1c6fcff1004190649k8c4f76e5s23830e7dd7d03823@mail.gmail.com>
Subject: Re: Regarding Cassandra Scalability
From: Gary Dusbabek <gdusbabek@gmail.com>
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Sun, Apr 18, 2010 at 11:14, dir dir <sikerasakti@gmail.com> wrote:
> Hi Gary,
>
>>The main reason is that the compaction operation (removing deleted
>>values) currently requires that an entire row be read into memory.
>
> Thank you for your explanation. But I still do not understand what do you
> mean.
>

When you delete a column in cassandra, the data is not really deleted.
 Instead a flag is turned on indicating the column is no longer valid
(we call it a 'tombstone').  During compaction the column family is
scanned and the tombstones are truly deleted.

> in my opinion, Actually the row contents must fit in available memory.
> if row contents are not fit in available memory, our software will raise
> exception out of memory. since it is true( "the row contents must fit in
> available memory"),
> then why you said that is a problem which it (Cassandra) cannot solved??
>

It was not correct of me to say it is a problem that cassandra cannot
solve.  Memory-efficient compactions will be addressed.

> You say: "compaction operation requires that entire row be read into memo=
ry"
> whether this is a problem of "out of memory"??=A0 When we need to perform
> compaction operation?? In what situation we shall perform compaction
> operation??

You will need to address the large rows yourself (consider breaking
them up).  You can identify these rows during compaction by setting
RowWarningThresholdInMB in storage-conf.xml.  When a big enough row
comes along, it is logged so you can go back later and address the
problem.

Regards,

Gary.

> Thank You.
>
> Dir.
>