From cassandra-user-return-720-apmail-incubator-cassandra-user-archive=incubator.apache.org@incubator.apache.org Tue Sep 29 15:16:45 2009 Return-Path: Delivered-To: apmail-incubator-cassandra-user-archive@minotaur.apache.org Received: (qmail 9916 invoked from network); 29 Sep 2009 15:16:45 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 29 Sep 2009 15:16:45 -0000 Received: (qmail 32997 invoked by uid 500); 29 Sep 2009 15:16:45 -0000 Delivered-To: apmail-incubator-cassandra-user-archive@incubator.apache.org Received: (qmail 32964 invoked by uid 500); 29 Sep 2009 15:16:45 -0000 Mailing-List: contact cassandra-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: cassandra-user@incubator.apache.org Delivered-To: mailing list cassandra-user@incubator.apache.org Received: (qmail 32955 invoked by uid 99); 29 Sep 2009 15:16:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Sep 2009 15:16:45 +0000 X-ASF-Spam-Status: No, hits=-1.8 required=10.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED X-Spam-Check-By: apache.org Received-SPF: unknown (athena.apache.org: error in processing during lookup of junrao@almaden.ibm.com) Received: from [32.97.182.138] (HELO e8.ny.us.ibm.com) (32.97.182.138) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Sep 2009 15:16:34 +0000 Received: from d01relay04.pok.ibm.com (d01relay04.pok.ibm.com [9.56.227.236]) by e8.ny.us.ibm.com (8.14.3/8.13.1) with ESMTP id n8TFDpW4014760 for ; Tue, 29 Sep 2009 11:13:51 -0400 Received: from d01av03.pok.ibm.com (d01av03.pok.ibm.com [9.56.224.217]) by d01relay04.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id n8TFGDPv257796 for ; Tue, 29 Sep 2009 11:16:13 -0400 Received: from d01av03.pok.ibm.com (loopback [127.0.0.1]) by d01av03.pok.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id n8TFGC9r025922 for ; Tue, 29 Sep 2009 11:16:12 -0400 Received: from d01ml604.pok.ibm.com (d01ml604.pok.ibm.com [9.56.227.90]) by d01av03.pok.ibm.com (8.12.11.20060308/8.12.11) with ESMTP id n8TFGAqK025741 for ; Tue, 29 Sep 2009 11:16:10 -0400 In-Reply-To: References: <23b1e84e0909241228x7b153481k7f0e135f4d1d3fc@mail.gmail.com> <23b1e84e0909270839ra1e6f2chb151885d1aa9e866@mail.gmail.com> <23b1e84e0909270918w5e1d7af3yf63570263b40a4e5@mail.gmail.com> <23b1e84e0909270920h4a5b1fb8m2bc7d004b95ec49d@mail.gmail.com> <23b1e84e0909281240u6266d359g1966e357118aff27@mail.gmail.com> <23b1e84e0909281632h1d1b7a25r16c7e1411c58164f@mail.gmail.com> Subject: Re: commit logs are not deleted X-KeepSent: E49B9727:B23B8ED8-88257640:0053B5EC; type=4; name=$KeepSent To: cassandra-user@incubator.apache.org X-Mailer: Lotus Notes Release 8.0.2 HF623 January 16, 2009 Message-ID: From: Jun Rao Date: Tue, 29 Sep 2009 08:16:09 -0700 X-MIMETrack: Serialize by Router on D01ML604/01/M/IBM(Release 8.5|December 05, 2008) at 09/29/2009 11:16:10 MIME-Version: 1.0 Content-type: multipart/alternative; Boundary="0__=07BBFCD3DFC0337C8f9e8a93df938690918c07BBFCD3DFC0337C" Content-Disposition: inline X-Virus-Checked: Checked by ClamAV on apache.org --0__=07BBFCD3DFC0337C8f9e8a93df938690918c07BBFCD3DFC0337C Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: quoted-printable That's true. However, all CL headers are cached in clHeaders and in mos= t cases, CL headers only need to be read, not written. Jun IBM Almaden Research Center K55/B1, 650 Harry Road, San Jose, CA 95120-6099 junrao@almaden.ibm.com Jonathan Ellis wrote on 09/28/2009 08:25:32 PM: > [image removed] > > Re: commit logs are not deleted > > Jonathan Ellis > > to: > > cassandra-user > > 09/28/2009 08:26 PM > > Please respond to cassandra-user > > > Right, but the corollary is that then you have to scan all the > commitlog segment headers when flushing an empty memtable. I think > the cure is potentially worse than the disease there. > > On Mon, Sep 28, 2009 at 10:21 PM, Jun Rao wr= ote: > > A solution to this problem is to still call discardCompletedSegment= s() when > > flushing an empty memtable. > > > > Jun > > IBM Almaden Research Center > > K55/B1, 650 Harry Road, San Jose, CA 95120-6099 > > > > junrao@almaden.ibm.com > > > > > > Igor Katkov wrote on 09/28/2009 04:32:06 PM: > > > >> [image removed] > >> > >> Re: commit logs are not deleted > >> > >> Igor Katkov > >> > >> to: > >> > >> cassandra-user > >> > >> 09/28/2009 04:33 PM > >> > >> Please respond to cassandra-user > > > >> > >> Excellent! Many thanks. > >> > >> It works, but there is a minor thing: > >> the very first created commit log segment does not get deleted. > >> Log if filled with lines like > >> Not safe to delete commit log d:/cassandra-4/data/commitlog > >> \CommitLog-1254177667321.log; dirty is 1, > >> > > > >> On Mon, Sep 28, 2009 at 5:21 PM, Jonathan Ellis wrote: > >> I have a fix now at > >> https://issues.apache.org/jira/browse/CASSANDRA-459 -- only applie= s to > >> current trunk. =A0When Jun finishes reviewing I will backport to 0= .4. > >> > >> Sorry for the wait, > >> > >> -Jonathan > >> > >> On Mon, Sep 28, 2009 at 2:40 PM, Igor Katkov wrote: > >> > Does anyone else has the same problem with 0.4? > >> > As far as I know Digg and Rackspace are biggest adopters of Cassandra. > >> > They > >> > must be running 0.3 then. > >> > > >> > > >> > On Sun, Sep 27, 2009 at 12:20 PM, Igor Katkov wrote: > >> >> > >> >> full log is here (new) > >> >> http://www.katkovonline.com/cassandra-log.zip > >> >> > >> >> On Sun, Sep 27, 2009 at 12:18 PM, Igor Katkov > >> >> wrote: > >> >>> > >> >>> There is no log lines with "column family 1" > >> >>> There are only "column family 0" > >> >>> > >> >>> On Sun, Sep 27, 2009 at 12:02 PM, Jonathan Ellis > >> >>> wrote: > >> >>>> > >> >>>> we're looking for "column family 1" > >> >>>> > >> > > >> >= --0__=07BBFCD3DFC0337C8f9e8a93df938690918c07BBFCD3DFC0337C Content-type: text/html; charset=ISO-8859-1 Content-Disposition: inline Content-transfer-encoding: quoted-printable

That's true. However, all CL headers are cached in clHeaders and in = most cases, CL headers only need to be read, not written.

Jun
IBM Almaden Research Center
K55/B1, 650 Harry Road, San Jose, CA 95120-6099

junrao@almaden.ibm.com


Jonathan Ellis <jbellis@gmail.com> wrote on 09/28/2009 08:25:= 32 PM:

> [image removed]

>
> Re: commit logs are not deleted

>
> Jonathan Ellis

>
> to:

>
> cassandra-user

>
> 09/28/2009 08:26 PM

>
> Please respond to cassandra-user

>
>
> Right, but the corollary is that then you have to scan all the
= > commitlog segment headers when flushing an empty memtable.  I= think
> the cure is potentially worse than the disease there.
>
> On Mon, Sep 28, 2009 at 10:21 PM, Jun Rao <junrao@almaden.ibm.c= om> wrote:
> > A solution to this problem is to still call discardCompletedS= egments() when
> > flushing an empty memtable.
> >
> > Jun
> > IBM Almaden Research Center
> > K55/B1, 650 Harry Road, San Jose, CA 95120-6099
> >
> > junrao@almaden.ibm.com
> >
> >
> > Igor Katkov <ikatkov@gmail.com> wrote on 09/28/2009 04:= 32:06 PM:
> >
> >> [image removed]
> >>
> >> Re: commit logs are not deleted
> >>
> >> Igor Katkov
> >>
> >> to:
> >>
> >> cassandra-user
> >>
> >> 09/28/2009 04:33 PM
> >>
> >> Please respond to cassandra-user
> >
> >>
> >> Excellent! Many thanks.
> >>
> >> It works, but there is a minor thing:
> >> the very first created commit log segment does not get de= leted.
> >> Log if filled with lines like
> >> Not safe to delete commit log d:/cassandra-4/data/commitl= og
> >> \CommitLog-1254177667321.log; dirty is 1,
> >>
> >
> >> On Mon, Sep 28, 2009 at 5:21 PM, Jonathan Ellis <jbell= is@gmail.com> wrote:
> >> I have a fix now at
> >> https://issues.apache.org/jira/browse/CASSANDRA-459 -- only = applies to
> >> current trunk. =A0When Jun finishes reviewing I will back= port to 0.4.
> >>
> >> Sorry for the wait,
> >>
> >> -Jonathan
> >>
> >> On Mon, Sep 28, 2009 at 2:40 PM, Igor Katkov <ikatkov@= gmail.com> wrote:
> >> > Does anyone else has the same problem with 0.4?
> >> > As far as I know Digg and Rackspace are biggest adop= ters of Cassandra.
> >> > They
> >> > must be running 0.3 then.
> >> >
> >> >
> >> > On Sun, Sep 27, 2009 at 12:20 PM, Igor Katkov <ik= atkov@gmail.com> wrote:
> >> >>
> >> >> full log is here (new)
> >> >> http://www.katkovonline.com/cassandra-log.zip
> >> >>
> >> >> On Sun, Sep 27, 2009 at 12:18 PM, Igor Katkov &l= t;ikatkov@gmail.com>
> >> >> wrote:
> >> >>>
> >> >>> There is no log lines with "column fami= ly 1"
> >> >>> There are only "column family 0" > >> >>>
> >> >>> On Sun, Sep 27, 2009 at 12:02 PM, Jonathan E= llis <jbellis@gmail.com>
> >> >>> wrote:
> >> >>>>
> >> >>>> we're looking for "column family 1&= quot;
> >> >>>>
> >> >
> >> >
= --0__=07BBFCD3DFC0337C8f9e8a93df938690918c07BBFCD3DFC0337C--