Mailing-List: contact cassandra-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cassandra-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of ikatkov@gmail.com designates
 209.85.218.226 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=CNyOZZSBvyNmwTIfefYyYjq2fYMfiCm2xyA4KdCFLV2HC3Nk/UPWbgdaNYZeCLAE/4
         19vpaJvhHCLXJMcM182beSjOsageknoDaj/bW0G96LKbWLSfOF/4H9rbtVLPxgn+ipOh
         OQoatNFcRUoDgz877OOXt4yaeuMPRkuFNZm28=
MIME-Version: 1.0
In-Reply-To: <23b1e84e0909242007s7facb02evbbe60f80976dc59d@mail.gmail.com>
References: <23b1e84e0909241228x7b153481k7f0e135f4d1d3fc@mail.gmail.com>
	<e06563880909241722u5a9790feqb9d9a11be32672a7@mail.gmail.com>
	<23b1e84e0909242007s7facb02evbbe60f80976dc59d@mail.gmail.com>
From: Igor Katkov <ikatkov@gmail.com>
Date: Fri, 25 Sep 2009 14:33:36 -0400
Message-ID: <23b1e84e0909251133if47ba36la41545444e096b22@mail.gmail.com>
Subject: Re: commit logs are not deleted
To: cassandra-user@incubator.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I tried latest stable version 0.3 and commit logs segments are in fact dele=
ted.
Tried it again on 0.4 set  periodic flush to 1min
(FlushPeriodInMinutes=3D"1") =3D> it's all the same, files remains there
forever.

I also noticed that there are other implicit CFs, can these prevent
logs from being deleted?
DEBUG - adding Channels as 0
DEBUG - adding LocationInfo as 1
DEBUG - adding HintsColumnFamily as 2

On Thu, Sep 24, 2009 at 11:07 PM, Igor Katkov <ikatkov@gmail.com> wrote:
> in my case commit log segments are never deleted (unless I restart the se=
rver)
> so they grow and grow and eventually hosts is running out of space.
>
> Any ideas how to fix it?
>
> On Thu, Sep 24, 2009 at 8:22 PM, Jonathan Ellis <jbellis@gmail.com> wrote=
:
>> When all the data from a given commit log segment has been flushed as
>> sstables, that segment can be deleted. =A0So if you do a bunch of
>> inserts and then stop, it's normal to have some commitlogs around
>> indefinitely. =A0All CFs are flushed on server restart, and the log
>> segments can then be removed, or you can add a periodic flush to the
>> CF definition so it will flush even when there has not been any extra
>> activity.
>>
>> (This last part doesn't quite work as designed right now, but we're
>> working on a fix: https://issues.apache.org/jira/browse/CASSANDRA-455)
>>
>> -Jonathan
>>
>> On Thu, Sep 24, 2009 at 2:28 PM, Igor Katkov <ikatkov@gmail.com> wrote:
>>> Hi,
>>>
>>> I'm using Cassandra 0.4.0 rc2
>>>
>>> I can't make Cassandra to wipe commit logs. They just keep
>>> accumulating, no mater what settings I play with in the config file.
>>>
>>> I insert 200ooo keys. 1 CF, one column, value is 170kb, single Cassandr=
a node.
>>> MemtableSizeInMB =3D32
>>> MemtableObjectCountInMillions =3D 0.1
>>>
>>> What do I do wrong?
>>>
>>> Please correct me if I misunderstood how things work:
>>>
>>> as soon as I insert a key-column-value, it gets written to memory, as
>>> soon as [data size or # of object] (see the settings above) are
>>> reached mem gets flushed to a commit log file. The very fact that I
>>> have growing number of commit logs files tells me that this flushing
>>> does happen.
>>>
>>> Now, commit logs records has to be transferred to the data and index
>>> files, I'm sure it happens as well, since my data folder is also
>>> growing, I see a lot of *.db files there.
>>> According to
>>> http://perspectives.mvdirona.com/2009/02/07/FacebookCassandraArchitectu=
reAndDesign.aspx
>>> commit logs has to be wiped as soon as =A0all its column families pushe=
d to disk.
>>> This thing does NOT happen somehow, I have only one column family
>>> defined in the conf file.
>>>
>>> Conf file - http://www.katkovonline.com/storage-conf.xml
>>>
>>
>