cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <hiroyuki.watan...@barclayscapital.com>
Subject RE: Removal of old data files
Date Fri, 02 Sep 2011 13:01:23 GMT
 
I see. Thank you for helpful information 

Yuki



-----Original Message-----
From: Sylvain Lebresne [mailto:sylvain@datastax.com] 
Sent: Friday, September 02, 2011 3:40 AM
To: user@cassandra.apache.org
Subject: Re: Removal of old data files

On Fri, Sep 2, 2011 at 12:11 AM,  <hiroyuki.watanabe@barclayscapital.com> wrote:
> Yes, I see files with name like
>     Orders-g-6517-Compacted
>
> However, all of those file have a size of 0.
>
> Starting from Monday to Thurseday we have 5642 files for -Data.db, 
> -Filter.db and Statistics.db and only 128 -Compacted files.
> and all of -Compacted file has size of 0.
>
> Is this normal, or we are doing something wrong?

You are not doing something wrong. The -Compacted files are just marker, to indicate that
the -Data file corresponding (with the same number) are, in fact, compacted and will eventually
be removed. So those files will always have a size of 0.

--
Sylvain

>
>
> yuki
>
> ________________________________
> From: aaron morton [mailto:aaron@thelastpickle.com]
> Sent: Thursday, August 25, 2011 6:13 PM
> To: user@cassandra.apache.org
> Subject: Re: Removal of old data files
>
> If cassandra does not have enough disk space to create a new file it 
> will provoke a JVM GC which should result in compacted SStables that 
> are no longer needed been deleted. Otherwise they are deleted at some 
> time in the future.
> Compacted SSTables have a file written out with a "compacted" extension.
> Do you see compacted sstables in the data directory?
> Cheers.
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
> On 26/08/2011, at 2:29 AM, yuki watanabe wrote:
>
> We are using Cassandra 0.8.0 with 8 node ring and only one CF.
> Every column has TTL of 86400 (24 hours). we also set 'GC grace 
> second' to 43200
> (12 hours).  We have to store massive amount of data for one day now 
> and eventually for five days if we get more disk space.
> Even for one day, we do run out disk space in a busy day.
>
> We run nodetool compact command at night or as necessary then we run 
> GC from jconsole. We observed that  GC did remove files but not 
> necessarily oldest ones.
> Data files from more than 36 hours ago and quite often three days ago 
> are still there.
>
> Does this behavior expected or we need adjust some other parameters?
>
>
> Yuki Watanabe
>
> _______________________________________________
>
>
>
> This e-mail may contain information that is confidential, privileged 
> or otherwise protected from disclosure. If you are not an intended 
> recipient of this e-mail, do not duplicate or redistribute it by any 
> means. Please delete it and any attachments and notify the sender that 
> you have received it in error. Unless specifically indicated, this 
> e-mail is not an offer to buy or sell or a solicitation to buy or sell 
> any securities, investment products or other financial product or 
> service, an official confirmation of any transaction, or an official 
> statement of Barclays. Any views or opinions presented are solely 
> those of the author and do not necessarily represent those of 
> Barclays. This e-mail is subject to terms available at the following 
> link: www.barcap.com/emaildisclaimer. By messaging with Barclays you 
> consent to the foregoing.  Barclays Capital is the investment banking 
> division of Barclays Bank PLC, a company registered in England (number
> 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.
> This email may relate to or be sent from other members of the Barclays 
> Group.
>
> _______________________________________________

Mime
View raw message