Mailing-List: contact cassandra-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cassandra-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of jbellis@gmail.com designates
 74.125.82.47 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=txp9VwTz5I0bmF5F9eWPCHPdfP5OkiPC1Xa1BKcmlvbL/EwBb+h0JUnZdxwr1z340X
         Vh4EfqbOLtCtBAzsqkYpBmwyu+k21RuNOhAdf9oYu1zDlGmyc9++IJSdR9HTBdY1fS/G
         NAdkO8lQY3fEQMRcfSfQbeto2aDudmonA3atY=
MIME-Version: 1.0
In-Reply-To: <ad2266761003130118r32952f37l120653c71d0a229c@mail.gmail.com>
References: <3b5f72031003111934m75e9de7w4616642b5a14a9fe@mail.gmail.com>
	<ff1a07b81003112328y2e4218a6mdfcc8ed565b947d9@mail.gmail.com>
	<ac68ab851003120027j1bae0e7fqb347d29eaeb667d@mail.gmail.com>
	<ad2266761003130118r32952f37l120653c71d0a229c@mail.gmail.com>
From: Jonathan Ellis <jbellis@gmail.com>
Date: Sat, 13 Mar 2010 07:35:56 -0600
Message-ID: <e06563881003130535r586fb14cqc5262e83d3148a81@mail.gmail.com>
Subject: Re: question about deleting from cassandra
To: cassandra-user@incubator.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

You should submit your minor change to jira for others who might want to tr=
y it.

On Sat, Mar 13, 2010 at 3:18 AM, Weijun Li <weijunli@gmail.com> wrote:
> Tried Sylvain's feature in 0.6 beta2 (need minor change) and it worked
> perfectly. Without this feature, as far as you have high volume new and
> expired columns your life will be miserable :-)
>
> Thanks for great job Sylvain!!
>
> -Weijun
>
> On Fri, Mar 12, 2010 at 12:27 AM, Sylvain Lebresne <sylvain@yakaz.com>
> wrote:
>>
>> I guess you can also vote for this ticket :
>> https://issues.apache.org/jira/browse/CASSANDRA-699 :)
>>
>> </advertising>
>>
>> --
>> Sylvain
>>
>>
>> On Fri, Mar 12, 2010 at 8:28 AM, Mark Robson <markxr@gmail.com> wrote:
>> > On 12 March 2010 03:34, Bill Au <bill.w.au@gmail.com> wrote:
>> >>
>> >> Let take Twitter as an example. =A0All the tweets are timestamped. =
=A0I
>> >> want
>> >> to keep only a month's worth of tweets for each user. =A0The number o=
f
>> >> tweets
>> >> that fit within this one month window varies from user to user. =A0Wh=
at
>> >> is the
>> >> best way to accomplish this?
>> >
>> > This is the "expiry" problem that has been discussed on this list
>> > before. As
>> > far as I can see there are no easy ways to do it with 0.5
>> >
>> > If you use the ordered partitioner and make the first part of the keys=
 a
>> > timestamp (or part of it) then you can get the keys and delete them.
>> >
>> > However, these deletes will be quite inefficient, currently each row
>> > must be
>> > deleted individually (there was a patch to range delete kicking around=
,
>> > I
>> > don't know if it's accepted yet)
>> >
>> > But even if range delete is implemented, it's still quite inefficient
>> > and
>> > not really what you want, and doesn't work with the RandomPartitioner
>> >
>> > If you have some metadata to say who tweeted within a given period (sa=
y
>> > 10
>> > days or 30 days) and you store the tweets all in the same key per user
>> > per
>> > period (say with one column per tweet, or use supercolumns), then you
>> > can
>> > just delete one key per user per period.
>> >
>> > One of the problems with using a time-based key with ordered partition=
er
>> > is
>> > that you're always going to have a data imbalance, so you may want to
>> > try
>> > hashing *part* of the key (The first part) so you can still range scan
>> > the
>> > next part. This may fix load balancing while still enabling you to use
>> > range
>> > scans to do data expiry.
>> >
>> > e.g. your key is
>> >
>> > Hash of day number + user id + timestamp
>> >
>> > Then you can range scan the entire day's tweets to expire them, and
>> > range
>> > scan a given user's tweets for a given day efficiently (and doing this
>> > for
>> > 30 days is just 30 range scans)
>> >
>> > Putting a hash in there fixes load balancing with OPP.
>> >
>> > Mark
>> >
>
>