cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Bromhead <...@instaclustr.com>
Subject Re: Deleted snapshot files filling up /var/lib/cassandra
Date Mon, 16 Mar 2015 23:12:21 GMT
Cassandra will by default snapshot your data directory on the following
events:

   - TRUNCATE and DROP schema events
   - when you run nodetool repair
   - when you run nodetool snapshot

Snapshots are just hardlinks to existing SSTables so the only disk space
they take up is for files that have since been compacted away. Disk space
for snapshots will be freed when the last link to the files are removed.
You can remove all snapshots in a cluster using nodetool clearsnapshot

Snapshots will fail if you are out of disk space (this is counterintuitive
to the above, but it is true), if you have not increased the number of
available file descriptors or if there are permissions issues.

Out of curiosity, how often are you running repair?

On 16 March 2015 at 15:52, David Wahler <dwahler@indeed.com> wrote:

> On Mon, Mar 16, 2015 at 5:28 PM, Jan <cnet62@yahoo.com> wrote:
> > David;
> >
> > all the packaged installations use the /var/lib/cassandra directory.
> > Could you check your yaml config files and see if you are using this
> default
> > directory  for backups
> >
> > May want to change it to a location with more disk space.
>
> We're using the default /var/lib/cassandra as our data directory,
> mounted as its own LVM volume. I don't see anything in cassandra.yaml
> about a backup directory. There is an "incremental_backups" option
> which is set to false.
>
> Increasing the available disk space doesn't really seem like a
> solution. We have only about 450MB of live data on the most
> heavily-loaded server, and the space taken up by these deleted files
> is growing by several GB per day. For now we can work around the
> problem by periodically restarting servers to close the file handles,
> but that hurts our availability and seems like a hack.
>



-- 

Ben Bromhead

Instaclustr | www.instaclustr.com | @instaclustr
<http://twitter.com/instaclustr> | (650) 284 9692

Mime
View raw message