cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Lebresne <sylv...@datastax.com>
Subject Re: Removal of old data files
Date Fri, 02 Sep 2011 07:40:27 GMT
On Fri, Sep 2, 2011 at 12:11 AM,  <hiroyuki.watanabe@barclayscapital.com> wrote:
> Yes, I see files with name like
>     Orders-g-6517-Compacted
>
> However, all of those file have a size of 0.
>
> Starting from Monday to Thurseday we have 5642 files for -Data.db,
> -Filter.db and Statistics.db and only 128 -Compacted files.
> and all of -Compacted file has size of 0.
>
> Is this normal, or we are doing something wrong?

You are not doing something wrong. The -Compacted files are just
marker, to indicate
that the -Data file corresponding (with the same number) are, in fact,
compacted and
will eventually be removed. So those files will always have a size of 0.

--
Sylvain

>
>
> yuki
>
> ________________________________
> From: aaron morton [mailto:aaron@thelastpickle.com]
> Sent: Thursday, August 25, 2011 6:13 PM
> To: user@cassandra.apache.org
> Subject: Re: Removal of old data files
>
> If cassandra does not have enough disk space to create a new file it will
> provoke a JVM GC which should result in compacted SStables that are no
> longer needed been deleted. Otherwise they are deleted at some time in the
> future.
> Compacted SSTables have a file written out with a "compacted" extension.
> Do you see compacted sstables in the data directory?
> Cheers.
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
> On 26/08/2011, at 2:29 AM, yuki watanabe wrote:
>
> We are using Cassandra 0.8.0 with 8 node ring and only one CF.
> Every column has TTL of 86400 (24 hours). we also set 'GC grace second' to
> 43200
> (12 hours).  We have to store massive amount of data for one day now and
> eventually for five days if we get more disk space.
> Even for one day, we do run out disk space in a busy day.
>
> We run nodetool compact command at night or as necessary then we run GC from
> jconsole. We observed that  GC did remove files but not necessarily oldest
> ones.
> Data files from more than 36 hours ago and quite often three days ago are
> still there.
>
> Does this behavior expected or we need adjust some other parameters?
>
>
> Yuki Watanabe
>
> _______________________________________________
>
>
>
> This e-mail may contain information that is confidential, privileged or
> otherwise protected from disclosure. If you are not an intended recipient of
> this e-mail, do not duplicate or redistribute it by any means. Please delete
> it and any attachments and notify the sender that you have received it in
> error. Unless specifically indicated, this e-mail is not an offer to buy or
> sell or a solicitation to buy or sell any securities, investment products or
> other financial product or service, an official confirmation of any
> transaction, or an official statement of Barclays. Any views or opinions
> presented are solely those of the author and do not necessarily represent
> those of Barclays. This e-mail is subject to terms available at the
> following link: www.barcap.com/emaildisclaimer. By messaging with Barclays
> you consent to the foregoing.  Barclays Capital is the investment banking
> division of Barclays Bank PLC, a company registered in England (number
> 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.
> This email may relate to or be sent from other members of the Barclays
> Group.
>
> _______________________________________________

Mime
View raw message