incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: Issue with nodetool clearsnapshot
Date Tue, 06 Mar 2012 09:03:58 GMT
> 1)Since you mentioned hard links, I would like to add that our data directory itself is
a sym-link. Could that be causing an issue ?
Seems unlikely. 

> I restarted the node and it went about deleting the files and the disk space has been
released. Can this be done using nodetool, and without restarting ?

Under 0.8.x they are deleted when the files are no longer in use and when JVM GC free's all
references. You can provoke this by getting the JVM to GC using JConsole or another JMX client.


If there is not enough disk free space GC is forced and free space reclaimed.  

Under 1.x file handles are counted and the files are quickly deleted. 

Cheers

-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 6/03/2012, at 7:38 AM, B R wrote:

> Hi Aaron,
> 
> 1)Since you mentioned hard links, I would like to add that our data directory itself
is a sym-link. Could that be causing an issue ?
> 
> 2)Yes, there are 0 byte files of the same numbers
> in Keyspace1 directory
> 0 Mar  4 01:33 Standard1-g-7317-Compacted
> 0 Mar  3 22:58 Standard1-g-7968-Compacted
> 0 Mar  3 23:10 Standard1-g-8778-Compacted
> 0 Mar  3 23:47 Standard1-g-8782-Compacted
> ...
> 
> I restarted the node and it went about deleting the files and the disk space has been
released. Can this be done using nodetool, and without restarting ?
> 
> Thanks.
> 
> On Mon, Mar 5, 2012 at 10:59 PM, aaron morton <aaron@thelastpickle.com> wrote:
>> It seems that instead of removing the snapshot, clearsnapshot moved the data files
from the snapshot directory to the parent directory and the size of the data for that keyspace
has doubled.
> That is not possible, there is only code there to delete a files in the snapshot. 
> 
> Note that in the snapshot are hard links to the files in the data dir. Deleting / clearing
the snapshot will not delete the files from the data dir if they are still in use. 
> 
>>  Many of the files are looking like duplicates.
>> 
>> in Keyspace1 directory
>> 156987786084 Jan 21 03:18 Standard1-g-7317-Data.db
>> 156987786084 Mar  4 01:33 Standard1-g-8850-Data.db
> Under 0.8.x files are not immediately deleted. Did the data directory contain zero size
-Compacted files with the same number ?
>   
> Cheers
> 
> 
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 5/03/2012, at 11:50 PM, B R wrote:
> 
>> Version 0.8.9
>> 
>> We run a 2 node cluster with RF=2. We ran a scrub and after that ran the clearsnapshot
to remove the backup snapshot created by scrub. It seems that instead of removing the snapshot,
clearsnapshot moved the data files from the snapshot directory to the parent directory and
the size of the data for that keyspace has doubled. Many of the files are looking like duplicates.
>> 
>> in Keyspace1 directory
>> 156987786084 Jan 21 03:18 Standard1-g-7317-Data.db
>> 156987786084 Mar  4 01:33 Standard1-g-8850-Data.db
>> 118211555728 Jan 31 12:50 Standard1-g-7968-Data.db
>> 118211555728 Mar  3 22:58 Standard1-g-8840-Data.db
>> 116902342895 Feb 25 02:04 Standard1-g-8832-Data.db
>> 116902342895 Mar  3 22:10 Standard1-g-8836-Data.db
>> 93788425710 Feb 21 04:20 Standard1-g-8791-Data.db
>> 93788425710 Mar  4 00:29 Standard1-g-8845-Data.db
>> .....
>> 
>> Even though the nodetool ring command shows the correct data size for the node, the
du -sh on the keyspace directory gives double the size.
>> 
>> Can you guide us to proceed from this situation ?
>> 
>> Thanks.
> 
> 


Mime
View raw message