Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 65625 invoked from network); 3 Feb 2011 06:13:15 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 3 Feb 2011 06:13:15 -0000 Received: (qmail 82796 invoked by uid 500); 3 Feb 2011 06:13:14 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 82332 invoked by uid 500); 3 Feb 2011 06:13:10 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 82324 invoked by uid 99); 3 Feb 2011 06:13:09 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 06:13:09 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [74.125.82.44] (HELO mail-ww0-f44.google.com) (74.125.82.44) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 06:13:02 +0000 Received: by wwa36 with SMTP id 36so820186wwa.25 for ; Wed, 02 Feb 2011 22:12:41 -0800 (PST) MIME-Version: 1.0 Received: by 10.216.156.149 with SMTP id m21mr2759767wek.22.1296713561283; Wed, 02 Feb 2011 22:12:41 -0800 (PST) Received: by 10.216.181.4 with HTTP; Wed, 2 Feb 2011 22:12:41 -0800 (PST) X-Originating-IP: [70.124.90.200] In-Reply-To: <5FAFCB6230A1754CBC58296E62D8D5E602E4BC6167@pa-ex-01.YOJOE.local> References: <5FAFCB6230A1754CBC58296E62D8D5E602E4BC613A@pa-ex-01.YOJOE.local> <2ecb62be-81b4-65b2-f1a7-eb47f341733f@me.com> <5FAFCB6230A1754CBC58296E62D8D5E602E4BC6167@pa-ex-01.YOJOE.local> Date: Thu, 3 Feb 2011 00:12:41 -0600 Message-ID: Subject: Re: rolling window of data From: Tyler Hobbs To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001485f45418321c56049b5aa723 X-Virus-Checked: Checked by ClamAV on apache.org --001485f45418321c56049b5aa723 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable No, Logsandra does not use a rolling window. The correct way to accomplish what you describe is the new (in 0.7) per-column TTL. Simply set this to 60 * 60 * 24 * 90 (90 day's worth of seconds) and your columns will magically disappear after that length of time. - Tyler On Wed, Feb 2, 2011 at 11:46 PM, Jeffrey Wang wrote: > Thanks for the link, but unfortunately it doesn=92t look like it uses a > rolling window. As far as I can tell, log entries just keep getting inser= ted > into Cassandra. > > > > -Jeffrey > > > > *From:* Aaron Morton [mailto:aaron@thelastpickle.com] > *Sent:* Wednesday, February 02, 2011 9:21 PM > *To:* user@cassandra.apache.org > *Subject:* Re: rolling window of data > > > > This project may provide some inspiration for you > https://github.com/thobbs/logsandra > > > > Not sure if it has a rolling window, if you find out let me know :) > > > > Aaron > > > > On 03 Feb, 2011,at 06:08 PM, Jeffrey Wang wrote: > > Hi, > > > > We=92re trying to use Cassandra 0.7 to store a rolling window of log data > (e.g. last 90 days). We use the timestamp of the log entries as the colum= n > names so we can do time range queries. Everything seems to be working fin= e, > but it=92s not clear if there is an efficient way to delete data that is = more > than 90 days old. > > > > Originally I thought that using a slice range on a deletion would do the > trick, but that apparently is not supported yet. Another idea I had was t= o > store the timestamp of the log entry as Cassandra=92s timestamp and pass = in > artificial timestamps to remove (thrift API), but that seems hacky. Does > anyone know if there is a good way to support this kind of rolling window= of > data efficiently? Thanks. > > > > -Jeffrey > > > > --001485f45418321c56049b5aa723 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable No, Logsandra does not use a rolling window.

The correct way to acco= mplish what you describe is the new (in 0.7) per-column TTL.=A0 Simply set = this to 60 * 60 * 24 * 90 (90 day's worth of seconds) and your columns = will magically disappear after that length of time.

- Tyler

On Wed, Feb 2, 2011 at 11:46 = PM, Jeffrey Wang <jwang@palantir.com> wrote:

Thanks for th= e link, but unfortunately it doesn=92t look like it uses a rolling window. = As far as I can tell, log entries just keep getting inserted into Cassandra= .

=A0

-Jeffrey

=A0

From: Aaron Morto= n [mailto:aaro= n@thelastpickle.com]
Sent: Wednesday, February 02, 2011 9:21 PM
To: user@cassandra.apache.o= rg
Subject: Re: rolling window of data

=A0

This project may provide some inspiration for you=A0https://github.= com/thobbs/logsandra

=A0

= Not sure if it has a rolling window, if you find out let me know :)=A0

<= /div>

=A0

A= aron



On 03 Fe= b, 2011,at 06:08 PM, Jeffrey Wang <jwang@palantir.com> wrote:

Hi,

=A0

We=92re trying to use Cassandra 0.7 to store a rolling wi= ndow of log data (e.g. last 90 days). We use the timestamp of the log entri= es as the column names so we can do time range queries. Everything seems to= be working fine, but it=92s not clear if there is an efficient way to dele= te data that is more than 90 days old.

=A0

Originally I thought t= hat using a slice range on a deletion would do the trick, but that apparent= ly is not supported yet. Another idea I had was to store the timestamp of t= he log entry as Cassandra=92s timestamp and pass in artificial timestamps t= o remove (thrift API), but that seems hacky. Does anyone know if there is a= good way to support this kind of rolling window of data efficiently? Thank= s.

=A0

-Jeffrey

=A0

--001485f45418321c56049b5aa723--