Return-Path: Delivered-To: apmail-incubator-cassandra-user-archive@minotaur.apache.org Received: (qmail 64980 invoked from network); 10 Mar 2010 23:54:08 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 10 Mar 2010 23:54:08 -0000 Received: (qmail 21366 invoked by uid 500); 10 Mar 2010 23:53:36 -0000 Delivered-To: apmail-incubator-cassandra-user-archive@incubator.apache.org Received: (qmail 21322 invoked by uid 500); 10 Mar 2010 23:53:36 -0000 Mailing-List: contact cassandra-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: cassandra-user@incubator.apache.org Delivered-To: mailing list cassandra-user@incubator.apache.org Received: (qmail 21314 invoked by uid 99); 10 Mar 2010 23:53:36 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Mar 2010 23:53:36 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of weijunli@gmail.com designates 209.85.221.193 as permitted sender) Received: from [209.85.221.193] (HELO mail-qy0-f193.google.com) (209.85.221.193) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Mar 2010 23:53:29 +0000 Received: by qyk31 with SMTP id 31so8167347qyk.8 for ; Wed, 10 Mar 2010 15:53:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=qyKVDr0lOrXPWFWTkIUj+cfvMNGD+L4UzeOJuaLVKoc=; b=rjsBcZkk6feUTCK8b7FZ3bg7nfbI1SikxhoymCq6V5IeB3keEFmxu1s8QCm6J1AuwS qPm4VnZvETAOggfECB3VZUCohK/BQJj/nN3EOGyeulcshvTxMQrrKJn3oJKExoqMAYBn N8S33s9RH7tL6kUqUfXZp3lhHkzpJXGH09Grw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=gWa9hdGrxt2TbDAFMS69mqpL0MKx8LesxnEPyWQZUjbIGKeKQW6Xu5mf84g/K6Muxa fvPXG8jj9OFu/dcpbkDIC6lscciYS3LnxYkXpwjKA9pB/2vRzX9mqIICckOHtk2iq70Z oXvVznW7tPc92VGw3ThqwNJoOdgwv3kRPL5mQ= MIME-Version: 1.0 Received: by 10.220.125.104 with SMTP id x40mr208264vcr.33.1268265187729; Wed, 10 Mar 2010 15:53:07 -0800 (PST) In-Reply-To: References: <4b8376b4.0603c00a.0ba2.739f@mx.google.com> <022f01cab463$3a2cd8a0$ae8689e0$@com> <007601cab5e7$b5e1fe00$21a5fa00$@com> <00c701cab6bc$52990f70$f7cb2e50$@com> Date: Wed, 10 Mar 2010 15:53:07 -0800 Message-ID: Subject: Re: Strategy to delete/expire keys in cassandra From: Weijun Li To: Sylvain Lebresne Cc: cassandra-user@incubator.apache.org Content-Type: multipart/alternative; boundary=001636c933e9fee22e04817affbc X-Virus-Checked: Checked by ClamAV on apache.org --001636c933e9fee22e04817affbc Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Never mind. Figured out I forgot to compile thrift :) Thanks, -Weijun On Wed, Mar 10, 2010 at 1:43 PM, Weijun Li wrote: > Hi Sylvain, > > I applied your patch to 0.5 but it seems that it's not compilable: > > 1) column.getTtl() is no defined in RowMutation.java > public static RowMutation getRowMutation(String table, String key, > Map> cfmap) > { > RowMutation rm =3D new RowMutation(table, key.trim()); > for (Map.Entry> entry : > cfmap.entrySet()) > { > String cfName =3D entry.getKey(); > for (ColumnOrSuperColumn cosc : entry.getValue()) > { > if (cosc.column =3D=3D null) > { > assert cosc.super_column !=3D null; > for (org.apache.cassandra.service.Column column : > cosc.super_column.columns) > { > rm.add(new QueryPath(cfName, > cosc.super_column.name, column.name), column.value, column.timestamp, > column.getTtl()); > } > } > else > { > assert cosc.super_column =3D=3D null; > rm.add(new QueryPath(cfName, null, cosc.column.name), > cosc.column.value, cosc.column.timestamp, cosc.column.getTtl()); > } > } > } > return rm; > } > > 2) CassandraServer.java: Column.setTtl() is not defined. > if (column instanceof ExpiringColumn) > { > thrift_column.setTtl(((ExpiringColumn) > column).getTimeToLive()); > } > > 3) CliClient.java: type mismatch for ColumnParent > thriftClient_.insert(tableName, key, new ColumnParent(columnFamily, > superColumnName), > new Column(columnName, value.getBytes(), > System.currentTimeMillis()), ConsistencyLevel.ONE); > > It seems that the patch doesn't add getTtl()/setTtl() stuff to Column.jav= a? > > > Thanks, > -Weijun > > -----Original Message----- >> From: Sylvain Lebresne [mailto:sylvain@yakaz.com] >> Sent: Thursday, February 25, 2010 2:23 AM >> To: Weijun Li >> Cc: cassandra-user@incubator.apache.org >> Subject: Re: Strategy to delete/expire keys in cassandra >> >> Hi, >> >> > Should I just run command (in Cassandra 0.5 source folder?) like: >> > patch =96p1 =96i 0001-Add-new-ExpiringColumn-class.patch >> > for all of the five patches in your ticket? >> >> Well, actually I lied. The patches were made for a version a little afte= r >> 0.5. >> If you really want to try, I attach a version of those patches that >> (should) >> work with 0.5 (There is only the 3 first patch, but the fourth one is fo= r >> tests so not necessary per se). Apply them with your patch command. >> Still, to compile that you will have to regenerate the thrift java >> interface >> (with ant gen-thrift-java), but for that you will have to install the >> right >> svn revision of thrift (which is libthrift-r820831 for 0.5). And if you >> manage to make it work, you will have to digg in cassandra.thrift as it >> make >> change to it. >> >> In the end, remember that this is not an official patch yet and it *will >> not* make it in Cassandra in its current form. All I can tell you is tha= t >> I >> need those expiring columns for quite some of my usage and I will do wha= t >> I >> can to make this feature included if and when possible. >> >> > Also what=92s your opinion on extending ExpiringColumn to expire a key >> > completely? Otherwise it will be difficult to track what are expired >> > or old rows in Cassandra. >> >> I'm not sure how to make full rows (or even full superColumns for that >> matter) expire. What if you set a row to expire after some time and add >> new >> columns before this expiration ? Should you update the expiration of the >> row >> ? Which is to say that a row will expires when it's last column expire, >> which is almost what you get with expiring column. >> The one thing you may want though is that when all the columns of a row >> expire (or, to be precise, get physically deleted), the row itself is >> deleted. Looking at the code, I'm not convince this happen and I'm not >> sure >> why. >> >> -- >> Sylvain >> >> > > --001636c933e9fee22e04817affbc Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Never mind. Figured out I forgot to compile thrift :)

Thanks,
-Weijun

On Wed, Mar 10, 2010 at 1:43 PM= , Weijun Li <wei= junli@gmail.com> wrote:
Hi Sylvain,
I applied your patch to 0.5 but it seems that it's not compilable:
1) column.getTtl() is no defined in RowMutation.java
public static R= owMutation getRowMutation(String table, String key, Map<String, List<= ColumnOrSuperColumn>> cfmap)
=A0=A0=A0 {
=A0=A0=A0=A0=A0=A0=A0 RowMutation rm =3D new RowMutation(tab= le, key.trim());
=A0=A0=A0=A0=A0=A0=A0 for (Map.Entry<String, List<= ;ColumnOrSuperColumn>> entry : cfmap.entrySet())
=A0=A0=A0=A0=A0= =A0=A0 {
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 String cfName =3D entry.getKe= y();
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 for (ColumnOrSuperColumn cosc : entry.get= Value())
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 {
=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 if (cosc.column =3D=3D null)
=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 {
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 assert cosc.super_column !=3D null;
=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 for (org.apache.cassandra.service.Colu= mn column : cosc.super_column.columns)
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 {
=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 rm.add(new QueryP= ath(cfName, cos= c.super_column.name, c= olumn.name), column.value, column.timestamp, column.getTtl());
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 }
=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 }
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 else
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 {
=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 assert cosc.super_colum= n =3D=3D null;
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= rm.add(new QueryPath(cfName, null, cosc.column.name), cosc.column.value, cosc.column.timest= amp, cosc.column.getTtl());
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 }
=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 }
=A0=A0=A0=A0=A0=A0=A0 }
=A0=A0=A0=A0=A0=A0=A0 return rm;<= br>=A0=A0=A0 }

2) CassandraServer.java: Column.setTtl() is not defin= ed.
if (column instanceof ExpiringColumn)
=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0 {
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 thrift_column.= setTtl(((ExpiringColumn) column).getTimeToLive());
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 }

3) CliClient.java: type mismatch= for ColumnParent
thriftClient_.insert(tableName, key, new ColumnParent(= columnFamily, superColumnName),
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 new Column(columnName, value.= getBytes(), System.currentTimeMillis()), ConsistencyLevel.ONE);

It seems that the patch doesn't add getTtl()/setTtl() stuff to Colu= mn.java?


Thanks,
-Weijun
-----Original Message-----
From: Sylvain Lebresne [mailto:sylvain@yakaz.com]
Sent: Thursday, February 25, 2010 2:23 AM
To: Weijun Li
Cc: cassandra-user@incubator.apache.org
Subject: Re: Strategy to delete/expire keys in cassandra

Hi,

> Should I just run command (in Cassandra 0.5 source folder?) like:
> patch =96p1 =96i =A00001-Add-new-ExpiringColumn-class.patch
> for all of the five patches in your ticket?

Well, actually I lied. The patches were made for a version a little after 0.5.
If you really want to try, I attach a version of those patches that (should= )
work with 0.5 (There is only the 3 first patch, but the fourth one is for tests so not necessary per se). Apply them with your patch command.
Still, to compile that you will have to regenerate the thrift java interfac= e
(with ant gen-thrift-java), but for that you will have to install the right=
svn revision of thrift (which is libthrift-r820831 for 0.5). And if you
manage to make it work, you will have to digg in cassandra.thrift as it mak= e
change to it.

In the end, remember that this is not an official patch yet and it *will not* make it in Cassandra in its current form. All I can tell you is that I=
need those expiring columns for quite some of my usage and I will do what I=
can to make this feature included if and when possible.

> Also what=92s your opinion on extending ExpiringColumn to expire a key=
> completely? Otherwise it will be difficult to track what are expired > or old rows in Cassandra.

I'm not sure how to make full rows (or even full superColumns for that<= br> matter) expire. What if you set a row to expire after some time and add new=
columns before this expiration ? Should you update the expiration of the ro= w
? Which is to say that a row will expires when it's last column expire,=
which is almost what you get with expiring column.
The one thing you may want though is that when all the columns of a row
expire (or, to be precise, get physically deleted), the row itself is
deleted. Looking at the code, I'm not convince this happen and I'm = not sure
why.

--
Sylvain




--001636c933e9fee22e04817affbc--