cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: commit logs are not deleted
Date Fri, 25 Sep 2009 18:57:37 GMT
https://issues.apache.org/jira/browse/CASSANDRA-455 will address
FlushPeriod not working.

On Fri, Sep 25, 2009 at 1:33 PM, Igor Katkov <ikatkov@gmail.com> wrote:
> I tried latest stable version 0.3 and commit logs segments are in fact deleted.
> Tried it again on 0.4 set  periodic flush to 1min
> (FlushPeriodInMinutes="1") => it's all the same, files remains there
> forever.
>
> I also noticed that there are other implicit CFs, can these prevent
> logs from being deleted?
> DEBUG - adding Channels as 0
> DEBUG - adding LocationInfo as 1
> DEBUG - adding HintsColumnFamily as 2
>
> On Thu, Sep 24, 2009 at 11:07 PM, Igor Katkov <ikatkov@gmail.com> wrote:
>> in my case commit log segments are never deleted (unless I restart the server)
>> so they grow and grow and eventually hosts is running out of space.
>>
>> Any ideas how to fix it?
>>
>> On Thu, Sep 24, 2009 at 8:22 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
>>> When all the data from a given commit log segment has been flushed as
>>> sstables, that segment can be deleted.  So if you do a bunch of
>>> inserts and then stop, it's normal to have some commitlogs around
>>> indefinitely.  All CFs are flushed on server restart, and the log
>>> segments can then be removed, or you can add a periodic flush to the
>>> CF definition so it will flush even when there has not been any extra
>>> activity.
>>>
>>> (This last part doesn't quite work as designed right now, but we're
>>> working on a fix: https://issues.apache.org/jira/browse/CASSANDRA-455)
>>>
>>> -Jonathan
>>>
>>> On Thu, Sep 24, 2009 at 2:28 PM, Igor Katkov <ikatkov@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I'm using Cassandra 0.4.0 rc2
>>>>
>>>> I can't make Cassandra to wipe commit logs. They just keep
>>>> accumulating, no mater what settings I play with in the config file.
>>>>
>>>> I insert 200ooo keys. 1 CF, one column, value is 170kb, single Cassandra
node.
>>>> MemtableSizeInMB =32
>>>> MemtableObjectCountInMillions = 0.1
>>>>
>>>> What do I do wrong?
>>>>
>>>> Please correct me if I misunderstood how things work:
>>>>
>>>> as soon as I insert a key-column-value, it gets written to memory, as
>>>> soon as [data size or # of object] (see the settings above) are
>>>> reached mem gets flushed to a commit log file. The very fact that I
>>>> have growing number of commit logs files tells me that this flushing
>>>> does happen.
>>>>
>>>> Now, commit logs records has to be transferred to the data and index
>>>> files, I'm sure it happens as well, since my data folder is also
>>>> growing, I see a lot of *.db files there.
>>>> According to
>>>> http://perspectives.mvdirona.com/2009/02/07/FacebookCassandraArchitectureAndDesign.aspx
>>>> commit logs has to be wiped as soon as  all its column families pushed to
disk.
>>>> This thing does NOT happen somehow, I have only one column family
>>>> defined in the conf file.
>>>>
>>>> Conf file - http://www.katkovonline.com/storage-conf.xml
>>>>
>>>
>>
>

Mime
View raw message