Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 96001 invoked from network); 8 Feb 2011 20:24:38 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 8 Feb 2011 20:24:38 -0000 Received: (qmail 11766 invoked by uid 500); 8 Feb 2011 20:24:36 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 11724 invoked by uid 500); 8 Feb 2011 20:24:35 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 11716 invoked by uid 99); 8 Feb 2011 20:24:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Feb 2011 20:24:35 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of kallin.nagelberg@gmail.com designates 209.85.213.172 as permitted sender) Received: from [209.85.213.172] (HELO mail-yx0-f172.google.com) (209.85.213.172) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Feb 2011 20:24:29 +0000 Received: by yxt33 with SMTP id 33so2774452yxt.31 for ; Tue, 08 Feb 2011 12:24:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type:content-transfer-encoding; bh=5CbL2gZo1uT1Kd7z5hOqNIJ1+LU+pxdXGHZcgE+hTgQ=; b=svreuQ6vHROgcPZJdCkadOZ9DNCBKAwYsQ0NWdSY8SMsOk6pSzPE6tb1OIo5ioU0TX qV1nqXZYFNtr8QQd7Tn9x/H/HuIb74HIN0q7eGVbBkY7cfCCfkS96QeIZJMx9xsQ6b2w mqoD3qB4CndMnMv+RCM0kGNYcw8ih9h7Yx+4Y= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=PNRv2pjlGRZ9PHGbAez/KUt4orGl6HAcxE6xDBLbSsgBnM8KnU192wJ+Wje5G5gwl8 ebjutNbS1fX6i3T4/Auvp8M244drPvrc5Hbytdopm3GVyRxgjzCTbU+y5SZjuQMfzG3p KnBlm51cUrrWZ05FWhlAe5alsC4EmAvl6N/Nk= Received: by 10.90.114.10 with SMTP id m10mr402329agc.111.1297196648190; Tue, 08 Feb 2011 12:24:08 -0800 (PST) MIME-Version: 1.0 Received: by 10.90.66.2 with HTTP; Tue, 8 Feb 2011 12:23:47 -0800 (PST) In-Reply-To: <05CEA178DD88EE4FA89EED77C245F8490C2BFB23@msex85.morningstar.com> References: <05CEA178DD88EE4FA89EED77C245F8490C2BFB23@msex85.morningstar.com> From: Kallin Nagelberg Date: Tue, 8 Feb 2011 15:23:47 -0500 Message-ID: Subject: Re: time to live rows To: user@cassandra.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org I did read those articles, but I didn't know know that deleting all the columns on a row was equivalent to deleting the row. Like I mentioned, I did delete all the columns from all my rows and then forced compaction before and after gc_grace had passed, but all the rows still exist. If they never disappear, then won't I run out of resources eventually? -Kal On Tue, Feb 8, 2011 at 3:09 PM, Jeremiah Jordan wrote: > You will have the same problem. =A0You just have to learn to ignore empty= rows when you query data. =A0See articles on delete mentioned earlier. > >>>> >> >>> > http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives >>>> >> >>> > http://wiki.apache.org/cassandra/FAQ#range_ghosts > > -----Original Message----- > From: Kallin Nagelberg [mailto:kallin.nagelberg@gmail.com] > Sent: Tuesday, February 08, 2011 1:36 PM > To: user@cassandra.apache.org > Subject: Re: time to live rows > > I'm thinking if this row expiry notion doesn't pan out then I might > create a 'lastAccessed' column with a secondary index (i think that's > right) on it. Then I can periodically run a query to find all > lastAccessed columns less than a certain value and manually delete > them. Sound reasonable? > > -Kal > > On Tue, Feb 8, 2011 at 12:09 PM, Kallin Nagelberg > wrote: >> Yes I did, on the org.apache.cassandra.db.ColumnFamilies.Main.Session ob= ject. >> >> -Kal >> >> On Tue, Feb 8, 2011 at 12:00 PM, Sylvain Lebresne = wrote: >>> Did you force a major compaction (with jconsole or nodetool) after gc_g= race >>> has elapsed ? >>> On Tue, Feb 8, 2011 at 5:46 PM, Kallin Nagelberg >>> wrote: >>>> >>>> Thanks, gc_grace works in the CLI. >>>> >>>> However, I'm not observing the desired effect. I am setting TTL on a >>>> single column in my column family, and I see the columns disappear >>>> when using 'list Session' (my columnfamily) in the CLI. I created the >>>> column family with gc_grace =3D 60, and after observing for a few >>>> minutes I am still seeing all the rows come back, none of them with >>>> columns. I was hoping the GC would delete the empty rows. >>>> >>>> -Kal >>>> >>>> On Tue, Feb 8, 2011 at 11:39 AM, Sylvain Lebresne >>>> wrote: >>>> > Not very logically,=A0It's actually gc_grace, not gc_grace_seconds i= n the >>>> > CLI. >>>> > >>>> > On Tue, Feb 8, 2011 at 5:34 PM, Kallin Nagelberg >>>> > wrote: >>>> >> >>>> >> I'm trying to set the gc_grace_seconds column family parameter but = no >>>> >> luck.. I got the name of it from the comment in cassandra.yaml: >>>> >> >>>> >> # =A0 =A0 - gc_grace_seconds: specifies the time to wait before gar= bage >>>> >> # =A0 =A0 =A0 =A0collecting tombstones (deletion markers). defaults= to 864000 >>>> >> (10 >>>> >> # =A0 =A0 =A0 =A0days). See http://wiki.apache.org/cassandra/Distri= butedDeletes >>>> >> >>>> >> create column family Session >>>> >> =A0 =A0with comparator =3D UTF8Type >>>> >> =A0 =A0and keys_cached =3D 10000 >>>> >> =A0 =A0and memtable_flush_after =3D 1440 >>>> >> =A0 =A0and memtable_throughput =3D 32 >>>> >> =A0 =A0 =A0 =A0and gc_grace_seconds =3D 60; >>>> >> >>>> >> error is 'No enum const class >>>> >> >>>> >> >>>> >> org.apache.cassandra.cli.CliUserHelp$ColumnFamilyArgument.GC_GRACE_= SECONDS'. >>>> >> >>>> >> Thanks, >>>> >> -Kal >>>> >> >>>> >> On Tue, Feb 8, 2011 at 11:02 AM, Sylvain Lebresne >>>> >> >>>> >> wrote: >>>> >> >> I hope you don't consider this a hijack of the thread... >>>> >> >> >>>> >> >> What I'd like to know is the following: >>>> >> >> >>>> >> >> The GC removes TTL rows some time after they expire, at its >>>> >> >> convenience. >>>> >> >> But will they stop being returned as soon as they expire? (This = is >>>> >> >> the >>>> >> >> expected behavior...) >>>> >> > >>>> >> > It is the individual column that have TTL. When a column expires,= it >>>> >> > becomes >>>> >> > a delete tombstone. Now, a row with tombstones (even only them) w= ill >>>> >> > show >>>> >> > during range request. But the explanation is >>>> >> > here:=A0http://wiki.apache.org/cassandra/FAQ#range_ghosts >>>> >> > >>>> >> >> >>>> >> >> On Tue, Feb 8, 2011 at 5:11 PM, Kallin Nagelberg >>>> >> >> wrote: >>>> >> >>> >>>> >> >>> So the empty row will be ultimately removed then? Is there a wa= y to >>>> >> >>> for the GC to verify this? >>>> >> >>> >>>> >> >>> Thanks, >>>> >> >>> -Kal >>>> >> >>> >>>> >> >>> On Tue, Feb 8, 2011 at 2:21 AM, Stu Hood wr= ote: >>>> >> >>> > The expired columns were converted into tombstones, which wil= l >>>> >> >>> > live >>>> >> >>> > for >>>> >> >>> > the >>>> >> >>> > GC timeout. The "empty" row will be cleaned up when those >>>> >> >>> > tombstones >>>> >> >>> > are >>>> >> >>> > removed. >>>> >> >>> > Returning the empty row is unfortunate... we'd love to find a >>>> >> >>> > more >>>> >> >>> > appropriate solution that might not involve endless scanning. >>>> >> >>> > See >>>> >> >>> > http://wiki.apache.org/cassandra/FAQ#i_deleted_what_gives >>>> >> >>> > http://wiki.apache.org/cassandra/FAQ#range_ghosts >>>> >> >>> > >>>> >> >>> > On Mon, Feb 7, 2011 at 1:49 PM, Kallin Nagelberg >>>> >> >>> > wrote: >>>> >> >>> >> >>>> >> >>> >> I also tried forcing a major compaction on the column family >>>> >> >>> >> using >>>> >> >>> >> JMX >>>> >> >>> >> but the row remains. >>>> >> >>> >> >>>> >> >>> >> On Mon, Feb 7, 2011 at 4:43 PM, Kallin Nagelberg >>>> >> >>> >> wrote: >>>> >> >>> >> > I tried that but I still see the row coming back on a list >>>> >> >>> >> > in the CLI. My concern is that there will b= e a >>>> >> >>> >> > pointer >>>> >> >>> >> > to an empty row for all eternity. >>>> >> >>> >> > >>>> >> >>> >> > -Kal >>>> >> >>> >> > >>>> >> >>> >> > On Mon, Feb 7, 2011 at 4:38 PM, Aaron Morton >>>> >> >>> >> > >>>> >> >>> >> > wrote: >>>> >> >>> >> >> Deleting all the columns in a row via TTL has the same af= fect >>>> >> >>> >> >> as >>>> >> >>> >> >> deleting th >>>> >> >>> >> >> row, the data will physically by removed during compactio= n. >>>> >> >>> >> >> >>>> >> >>> >> >> Aaron >>>> >> >>> >> >> >>>> >> >>> >> >> >>>> >> >>> >> >> On 08 Feb, 2011,at 10:24 AM, Bill Speirs >>>> >> >>> >> >> >>>> >> >>> >> >> wrote: >>>> >> >>> >> >> >>>> >> >>> >> >> I don't think this is supported (but I could be completel= y >>>> >> >>> >> >> wrong). >>>> >> >>> >> >> However, I'd love to see this functionality as well. >>>> >> >>> >> >> >>>> >> >>> >> >> How would one go about requesting such a feature? >>>> >> >>> >> >> >>>> >> >>> >> >> Bill- >>>> >> >>> >> >> >>>> >> >>> >> >> On Mon, Feb 7, 2011 at 4:15 PM, Kallin Nagelberg >>>> >> >>> >> >> wrote: >>>> >> >>> >> >>> Hey, >>>> >> >>> >> >>> >>>> >> >>> >> >>> I have read about the new TTL columns in Cassandra 0.7. = In >>>> >> >>> >> >>> my >>>> >> >>> >> >>> case >>>> >> >>> >> >>> I'd >>>> >> >>> >> >>> like to expire an entire row automatically after a certa= in >>>> >> >>> >> >>> amount >>>> >> >>> >> >>> of >>>> >> >>> >> >>> time. Is this possible as well? >>>> >> >>> >> >>> >>>> >> >>> >> >>> Thanks, >>>> >> >>> >> >>> -Kal >>>> >> >>> >> >>> >>>> >> >>> >> >> >>>> >> >>> >> > >>>> >> >>> > >>>> >> >>> > >>>> >> >> >>>> >> > >>>> >> > >>>> > >>>> > >>> >>> >> >