incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artur Kronenberg <artur.kronenb...@openmarket.com>
Subject Re: Try to configure commitlog_archiving.properties
Date Tue, 10 Dec 2013 10:47:13 GMT
Hi,

There is some docs on the internet for this operations. It is basically 
as presented in the archive-commitlog file. 
(commitlog_archiving.properties).

The way the operations work: The operation is called automatically with 
parameters that give you control over what you want to do with it. It 
looks to me like your copy command for the archiving wants to copy every 
file. This is unnecessary. Cassandra will give you the arguments. E.g. 
Documentation for the archive command reads:

# Command to execute to archive a commitlog segment
# Parameters: %path => Fully qualified path of the segment to archive
#             %name => Name of the commit log.
# Example: archive_command=/bin/ln %path /backup/%name
#
# Limitation: *_command= expects one command with arguments. STDOUT
# and STDIN or multiple commands cannot be executed.  You might want
# to script multiple commands and add a pointer here.

Argument 1 will give you the path to the files you'd want to copy, while 
argument 2 will give uou the name if it. You can then create a command:

archive_command=/bin/bash /home/cassandra/scripts/cassandra-archive.sh 
%path %name

The above would be an example for me. As the commands by default only 
execute 1 command, I have them point to a custom script that does what I 
desire.

My script then looks something like this:

#! /bin/bash
# use bzip to compress the file
bzip2 --best -k $1
# move to commit log archive
mv $1.bz2 $HOME/commitlog_restore/$2.bz2

I compress my commitlog and then move it somewhere else. Cassandra will 
call this operation first, and then delete the commitlog.
  You can apply similar behaviours to all of those commands.

I would recommend to set up a cleanup scripts, otherwise your saved 
commit-log files will floud your harddrive. I would also test this 
locally first to make sure it works. For testing locally, bare in mind 
that you may wanna set your max file sizes to very low values as the 
command is only called when the commitlog is deleted. So having 
commitlogs that are 250 MB  or bigger will need a lot of writing to make 
the event trigger.

I hope I got that right and it helps.

Cheers.

FYI there was a bug with this, so you may wanna be on the right version:
https://issues.apache.org/jira/browse/CASSANDRA-5909

On 06/12/13 15:33, Vicky Kak wrote:
> >>Why, can you give me a good example and the good way to configure 
> archive
> >>commit logs ?
>
> Take a look at the cassandra code ;)
>
>
> On Fri, Dec 6, 2013 at 3:34 PM, Bonnet Jonathan 
> <jonathan.bonnet@externe.bnpparibas.com 
> <mailto:jonathan.bonnet@externe.bnpparibas.com>> wrote:
>
>     Hello,
>
>       I try to configure commitlog_archiving.properties to take
>     advantage of
>     backup and restore at a point of time, but there is no ressources on
>     internet for that. So i need some help.
>
>       If I understand I have 4 parameters:
>
>     archive_command=
>
>     restore_command=
>
>     restore_directories=
>
>     restore_point_in_time=
>
>       Forget for the moment the restore_point_in_time, should i put
>     one command
>     only for the archive_command; my wish is to copy all the commitlogs on
>     another directory:
>
>     restore_directories=/produits/cassandra/cassandra_data/archived_logs
>
>     archive_command=/bin/cp -
>     f /produits/cassandra/cassandra_commit/*.log
>     /produits/cassandra/cassandra_
>     data/archived_logs
>
>     restore_command=/bin/cp -
>     f /produits/cassandra/cassandra_data/archived_logs/*.log
>     /produits/cassandr
>     a/cassandra_commit
>
>     But it doesn't work when i restart the node which interest me, it
>     doesn't
>     copy anything.
>
>     Why, can you give me a good example and the good way to configure
>     archive
>     commit logs ? there's nothing on the net except Datastax Website.
>
>     Thanks for your answear.
>
>     Regards,
>
>     Bonnet Jonathan.
>
>


Mime
View raw message