cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynald Bourtembourg <>
Subject Re: [Marketing Mail] Re: [Marketing Mail] Cassandra 2.1: Snapshot data changing while transferring
Date Wed, 01 Jun 2016 12:20:12 GMT
Hi Paul,

If I understand correctly, you are making a tar file with all the 
folders named "snapshots" (i.e. the folder under which all the snapshots 
are created. So you have one /snapshots /folder per table).
If this is the case, when you are executing "nodetool repair", Cassandra 
will create a snapshot at the beginning of the repair, creating a new 
directory under each /snapshots/ directories. If this happens while you 
are creating your tar file, you will get the error you saw.

If you are not yet doing it, I advise you to use the -t option of the 
"nodetool snapshot" command to specify a specific name to your snapshot.
Then you should copy only the directories named 
snapshots/<your_specific_snapshot_name> in your tar file.

Can you confirm that you are creating your tar file from the snapshots 
directories directly (resulting in taking all the currently generated 

Kind regards


On 01/06/2016 13:27, Paul Dunkler wrote:
>> I guess this might come from the incremental repairs...
>> The repair time is stored in the sstable (RepairedAt timestamp metadata).
> By the way: We are not using incremental repairs at all. So can't be 
> the case here.
> It really seems like there is somewhat that can still change data in 
> snapshot directories. I feel like it's something to do with flushing / 
> compaction. But no clue, what... :(
>> Cheers,
>> Reynald
>> On 31/05/2016 11:03, Paul Dunkler wrote:
>>> Hi there,
>>> i am sometimes running in very strange errors while backing up 
>>> snapshots from a cassandra cluster.
>>> Cassandra version:
>>> 2.1.11
>>> What i basically do:
>>> 1. nodetool snapshot
>>> 2. tar all snapshot folders into one file
>>> 3. transfer them to another server
>>> What happens is that tar just sometimes give the error message "file 
>>> changed as we read it" while its adding a .db-file from the folder 
>>> of the previously created snapshot.
>>> If i understand everything correct, this SHOULD never happen. 
>>> Snapshots should be totally immutable, right?
>>> Am i maybe hitting a bug or is there some rare case with running 
>>> repair operations or what-so-ever which can change snapshotted data?
>>> I already searched through cassandra jira but couldn't find a bug 
>>> which looks related to this behaviour.
>>> Would love to get some help on this.
>>> —
>>> Paul Dunkler
> —
> Paul Dunkler

View raw message