incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <hiroyuki.watan...@barclayscapital.com>
Subject RE: Removal of old data files
Date Thu, 01 Sep 2011 22:11:22 GMT
Yes, I see files with name like
    Orders-g-6517-Compacted

However, all of those file have a size of 0.

Starting from Monday to Thurseday we have 5642 files for -Data.db, -Filter.db and Statistics.db
and only 128 -Compacted files.
and all of -Compacted file has size of 0.

Is this normal, or we are doing something wrong?


yuki


________________________________
From: aaron morton [mailto:aaron@thelastpickle.com]
Sent: Thursday, August 25, 2011 6:13 PM
To: user@cassandra.apache.org
Subject: Re: Removal of old data files

If cassandra does not have enough disk space to create a new file it will provoke a JVM GC
which should result in compacted SStables that are no longer needed been deleted. Otherwise
they are deleted at some time in the future.

Compacted SSTables have a file written out with a "compacted" extension.

Do you see compacted sstables in the data directory?

Cheers.

-----------------
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 26/08/2011, at 2:29 AM, yuki watanabe wrote:


We are using Cassandra 0.8.0 with 8 node ring and only one CF.
Every column has TTL of 86400 (24 hours). we also set 'GC grace second' to 43200
(12 hours).  We have to store massive amount of data for one day now and eventually for five
days if we get more disk space.
Even for one day, we do run out disk space in a busy day.

We run nodetool compact command at night or as necessary then we run GC from jconsole. We
observed that  GC did remove files but not necessarily oldest ones.
Data files from more than 36 hours ago and quite often three days ago are still there.

Does this behavior expected or we need adjust some other parameters?


Yuki Watanabe


_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected
from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or
redistribute it by any means. Please delete it and any attachments and notify the sender that
you have received it in error. Unless specifically indicated, this e-mail is not an offer
to buy or sell or a solicitation to buy or sell any securities, investment products or other
financial product or service, an official confirmation of any transaction, or an official
statement of Barclays. Any views or opinions presented are solely those of the author and
do not necessarily represent those of Barclays. This e-mail is subject to terms available
at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent
to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC,
a company registered in England (number 1026167) with its registered office at 1 Churchill
Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays
Group.
_______________________________________________

Mime
View raw message