cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: Incremental backups
Date Thu, 19 Jan 2012 18:40:56 GMT
mmm, they are not included in the snapshot they are probably not used.

Have you dropped an index call 09partition on AttractionCheckins?

In [52]: "".join(chr(int(x+y, 16)) for x,y in zip("3039706172746974696f6e"[0::2], "3039706172746974696f6e"[1::2]))
Out[52]: '09partition'

The simple thing to do is restart cassandra and grep the log for the file names. If they are
not there you are safe to blast away. 

your upgrade process was correct.

Cheers

-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 19/01/2012, at 11:34 PM, Michael Vaknine wrote:

> When I upgraded I did it in 2 stages.
>  
> Upgrade from 0.7.6 to 1.0.0
> Run scrub on each node.
> Run repair on the cluster
> Upgrade to 1.0.3
>  
> Is it safe to run scrub again? Because it did not seem to help when I updated it to 1.0.0
> Was there a bug in the scrub process in 1.0.0?
>  
> What is the best way to handle this to solve and get a valid environment?
>  
> Thanks
> Michael
>  
>  
> From: aaron morton [mailto:aaron@thelastpickle.com] 
> Sent: Thursday, January 19, 2012 12:26 PM
> To: user@cassandra.apache.org
> Subject: Re: Incremental backups
>  
> Did you run a scrub as part of the upgrade process ? That will re-write all the sstables
and remove the old ones.
>  
> If not run a scrub now and it will re-write the data with a -hb- format in the file name.

>  
> Cheers
>  
>  
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>  
> On 19/01/2012, at 4:38 PM, Michael Vaknine wrote:
> 
> 
> I am on 1.0.3 release and it looks like very old files that remained from the upgrade
process.
> How can I verify that?
>  
> Michael
>  
> From: aaron morton [mailto:aaron@thelastpickle.com] 
> Sent: Thursday, January 19, 2012 2:22 AM
> To: user@cassandra.apache.org
> Subject: Re: Incremental backups
>  
> Looks like you are on a 0.7.X release, which one exactly ? It would be a really good
idea to at least be on 8.X, preferably 1.0
>  
> Pre 1.0 compacted SSTables were removed during JVM GC, but compacted SSTables  have a
.Compacted file created so we know they are no longer needed. 
>  
> These SSTables look like secondary index files. It may be a bug if they are not included
in the incremental backups. 
>  
> Cheers 
>  
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>  
> On 19/01/2012, at 12:13 AM, Michael Vaknine wrote:
> 
> 
> 
> Hi,
> Thank you for response.
> I did restart for all the nodes and now I can see files in backup folders so It seems
like it is working.
> During this process I have noticed to something very strange
>  
> In data/City folder there are files that are not created in the snapshot folder (it looks
like old orphaned files)
> Is there any process of cassandta that will delete uneeded files I tried to run nodetool
cleanup but it did not help.
>  
> This is the files:
> -rw-r--r-- 1 cassandra cassandra     230281 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Data.db
> -rw-r--r-- 1 cassandra cassandra       1936 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Filter.db
> -rw-r--r-- 1 cassandra cassandra         27 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Index.db
> -rw-r--r-- 1 cassandra cassandra       4264 2011-12-06 00:57 AttractionCheckins.3039706172746974696f6e-f-157-Statistics.db
> -rw-r--r-- 1 cassandra cassandra       1321 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Data.db
> -rw-r--r-- 1 cassandra cassandra         16 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Filter.db
> -rw-r--r-- 1 cassandra cassandra         27 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Index.db
> -rw-r--r-- 1 cassandra cassandra       4264 2011-12-06 00:58 AttractionCheckins.3039706172746974696f6e-f-158-Statistics.db
> -rw-r--r-- 1 cassandra cassandra    2627100 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Data.db
> -rw-r--r-- 1 cassandra cassandra       1936 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Filter.db
> -rw-r--r-- 1 cassandra cassandra         20 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Index.db
> -rw-r--r-- 1 cassandra cassandra       4264 2011-12-06 06:55 Attractions.3039706172746974696f6e-f-1156-Statistics.db
> -rw-r--r-- 1 cassandra cassandra    2238358 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Data.db
> -rw-r--r-- 1 cassandra cassandra         16 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Filter.db
> -rw-r--r-- 1 cassandra cassandra         20 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Index.db
> -rw-r--r-- 1 cassandra cassandra       4264 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1157-Statistics.db
> -rw-r--r-- 1 cassandra cassandra         92 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Data.db
> -rw-r--r-- 1 cassandra cassandra         16 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Filter.db
> -rw-r--r-- 1 cassandra cassandra         20 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Index.db
> -rw-r--r-- 1 cassandra cassandra       4264 2011-12-06 07:50 Attractions.3039706172746974696f6e-f-1158-Statistics.db
> -rw-r--r-- 1 cassandra cassandra      44799 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Data.db
> -rw-r--r-- 1 cassandra cassandra       1936 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Filter.db
> -rw-r--r-- 1 cassandra cassandra        196 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Index.db
> -rw-r--r-- 1 cassandra cassandra       4264 2011-12-06 01:25 CityResources.3039706172746974696f6e-f-365-Statistics.db
> -rw-r--r-- 1 cassandra cassandra       7647 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Data.db
> -rw-r--r-- 1 cassandra cassandra         24 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Filter.db
> -rw-r--r-- 1 cassandra cassandra         96 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Index.db
> -rw-r--r-- 1 cassandra cassandra       4264 2011-12-06 07:50 CityResources.3039706172746974696f6e-f-366-Statistics.db
>  
>  
> Thanks
> Michael
>  
> From: Alain RODRIGUEZ [mailto:arodrime@gmail.com] 
> Sent: Wednesday, January 18, 2012 10:40 AM
> To: user@cassandra.apache.org
> Subject: Re: Incremental backups
>  
> As this option is in the cassandra.yaml file, you might need to perform a restart of
your entire cluster (a rolling restart should work).
>  
> Hope this will help.
>  
> Alain
> 
> 2012/1/18 Michael Vaknine <michaelv@citypath.com>
> Hi,
> I am configured to do incremental backups on all my node on the cluster but it is not
working.
> In cassandra.yaml : incremental_backups: true
> When I check data folder there are some keyspaces that has folder backups but empty and
I suspect this is a folder created in the past when I had 0.7.6 version.
> In a new creted Keyspace the folder does not exists.
> Does someone know if I need to configure any thing besides cassandra.yaml for this to
work?
>  
> Thanks
> Michael


Mime
View raw message