From user-return-30464-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Tue Dec 4 20:12:42 2012 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7C8F1D3B6 for ; Tue, 4 Dec 2012 20:12:42 +0000 (UTC) Received: (qmail 79275 invoked by uid 500); 4 Dec 2012 20:12:38 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 79244 invoked by uid 500); 4 Dec 2012 20:12:38 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 79233 invoked by uid 99); 4 Dec 2012 20:12:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Dec 2012 20:12:38 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a80.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Dec 2012 20:12:34 +0000 Received: from homiemail-a80.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a80.g.dreamhost.com (Postfix) with ESMTP id 9662837A08B for ; Tue, 4 Dec 2012 12:11:53 -0800 (PST) Received: from [192.168.0.134] (unknown [116.90.132.105]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a80.g.dreamhost.com (Postfix) with ESMTPSA id 14DE937A089 for ; Tue, 4 Dec 2012 12:11:49 -0800 (PST) From: aaron morton Content-Type: multipart/alternative; boundary="Apple-Mail=_EF1A1A7A-3FA1-44B1-A028-A72FB68BBE06" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: Data backup and restore Date: Wed, 5 Dec 2012 09:11:44 +1300 References: <50B5F81F.5020908@panasiangroup.com> <50BE228C.5060909@panasiangroup.com> To: user@cassandra.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_EF1A1A7A-3FA1-44B1-A028-A72FB68BBE06 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 I wrote a script to sym link the snapshots together the other day = https://github.com/amorton/cass_snapshot_link I've not really used it in anger yet. That is to say I wrote it for fun = and it worked on my mac book. If you use it let me know if it works.=20 Cheers A ----------------- Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 5/12/2012, at 6:51 AM, Alain RODRIGUEZ wrote: > Hi Adeel, >=20 > I am not sure this is the best solution but we did it this way: >=20 > On one production server : > - $cassandra-cli -f show_schema > schema (show_schema file contains = "use ; show schema;") > - Then open the schema file and remove headers lines, your file must = start by "create keyspace..."=20 >=20 > On one dev server > - Copy the schema file > - $cassandra-cli -f schema >=20 > Now you have your keyspace in your dev env. >=20 > if you have 1 node dev cluster and RF=3Dtotal number of node in = production, then simply snapshot whatever you want to restore (CF or all = keyspace) then copy these files in the CF directories of your test = cluster and refresh the new sstables. >=20 > if RF < total number of node in production then you need to take the = sstable from various nodes and take care not overriding files with the = same name while copying files. >=20 > I have some bash/parallel-ssh scripts to do this, but not on this = computer. >=20 > I would be glad learning how other people do this. >=20 > Alain >=20 >=20 > 2012/12/4 Yang > my guess (from what I learnt on this forum): you probably have to = manually create the schema on the new cluster. shutdown new cluster. = overwrite the column family files with your backup on all nodes in the = new cluster, then boot up. >=20 >=20 > On Tue, Dec 4, 2012 at 8:19 AM, Adeel Akbar = wrote: > Hi Tomas, >=20 > You are right and now my question is how I restore on test cluster. Do = I need to create column families and then copy snapshot on each = directory? >=20 >=20 > Thanks & Regards >=20 > Adeel Akbar >=20 > On 12/4/2012 9:09 PM, Tomas Nunez wrote: >> Hi >>=20 >> I think he was talking about the "fragmentation" of the snapshot. In = cassandra 1.0.X all ColumnFamilies are in the same directory, but in = cassandra 1.1.X each ColumnFamily is in its own directory, and snapshots = of each ColumnFamily are inside this directory. >>=20 >> 1.0.X Snapshot directory: >> /cassandra/data//snapshots/ >>=20 >> 1.1.X Snapshot directory >> = /cassandra/data///snapshots/= >>=20 >> In 1.0.X you can restore a Keyspace backup by copying just one = directory. In 1.1.X it seems you need to copy one directory for each = ColumnFamily, which is a little more complicated. >>=20 >>=20 >>=20 >> 2012/12/1 Tyler Hobbs >> The nodetool snapshot command has keyspace and column family options = (from nodetool --help): >>=20 >> snapshot [keyspaces...] -cf [columnfamilyName] -t [snapshotName] - = Take a snapshot of the optionally specified column family of the = specified keyspaces using optional name snapshotName >>=20 >>=20 >> On Wed, Nov 28, 2012 at 5:40 AM, Adeel Akbar = wrote: >> Dear All,=20 >>=20 >> I have Cassandra 1.1.4 cluster with 2 nodes. I need to take backup = and restore on staging for testing purpose. I have taken snapshot with = below mentioned command but It created snapshot on every Keyspace's = column family. Is there any other way to take backup and restore quick.=20= >>=20 >> /opt/apache-cassandra-1.1.4/bin/nodetool -h localhost snapshot -t = cassandra_bkup >>=20 >> Snapshot directory: >> /var/log/cassandra/data//>=20 >> --=20 >>=20 >> Thanks & Regards >>=20 >> Adeel Akbar >>=20 >>=20 >>=20 >>=20 >> --=20 >> Tyler Hobbs >> DataStax >>=20 >>=20 >>=20 >>=20 >> --=20 >> >> www.groupalia.com=09 >> Tom=E0s N=FA=F1ez >> IT-Sysprod >> Tel. + 34 93 159 31 00=20 >> Fax. + 34 93 396 18 52 >> Llull, 95-97, 2=BA planta, 08005 Barcelona >> Skype: tomas.nunez.groupalia >> tomas.nunez@groupalia.com >> Twitter Facebook = Linkedin >=20 >=20 >=20 --Apple-Mail=_EF1A1A7A-3FA1-44B1-A028-A72FB68BBE06 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 I = wrote a script to sym link the snapshots together the other day https://github.com/= amorton/cass_snapshot_link

I've not really used = it in anger yet. That is to say I wrote it for fun and it worked on my = mac book. If you use it let me know if it = works. 

Cheers
A

http://www.thelastpickle.com

On 5/12/2012, at 6:51 AM, Alain RODRIGUEZ <arodrime@gmail.com> = wrote:

Hi Adeel,

I am not sure this is the = best solution but we did it this way:

On one = production server :
- $cassandra-cli -f show_schema > = schema (show_schema file contains "use <myKeyspaceName>; show = schema;")
- Then open the schema file and remove headers lines, your = file must start by "create = keyspace..." 

On one dev = server
- Copy the schema file
- $cassandra-cli -f = schema

Now you have your keyspace in your dev = env.

if you have 1 node dev cluster and = RF=3Dtotal number of node in production, then simply snapshot whatever = you want to restore (CF or all keyspace) then copy these files in the CF = directories of your test cluster and refresh the new sstables.

if RF < total number of node in production = then you need to take the sstable from various nodes and take care not = overriding files with the same name while copying = files.

I have some bash/parallel-ssh scripts to = do this, but not on this computer.

I would be glad learning how other people do = this.

Alain


2012/12/4 Yang = <teddyyyy123@gmail.com>
my guess (from what I = learnt on this forum): you probably have to manually create the schema = on the new cluster. shutdown new cluster. overwrite the column family = files with your backup on all nodes in the new cluster, then boot = up.


On Tue, Dec 4, 2012 at 8:19 AM, Adeel = Akbar <adeel.akbar@panasiangroup.com> = wrote:
=20 =20 =20
Hi Tomas,

You are right and now my question is how I restore on test cluster. Do I need to create column families and then copy snapshot on each directory?

=20 =20 =20 =20 =20 =20 =20 =20 =20 =20


Thanks & Regards

Adeel Akbar

On 12/4/2012 9:09 PM, Tomas Nunez wrote:
Hi

I think he was talking about the "fragmentation" of the snapshot. In cassandra 1.0.X all ColumnFamilies are in the same directory, but in cassandra 1.1.X each ColumnFamily is in its own directory, and snapshots of each ColumnFamily are inside this directory.

1.0.X Snapshot directory:
=
/cassandra/data/<KeySpace>/snapshots/<cassandra_bkup_files&g= t;

1.1.X Snapshot directory
=
/cassandra/data/<KeySpace>/<ColumnFamily>/snapshots/<c= assandra_bkup_files>

In 1.0.X you can restore a Keyspace backup by copying just one directory. In 1.1.X it seems you need to copy one directory for each ColumnFamily, which is a little more complicated.



2012/12/1 Tyler Hobbs <tyler@datastax.com>
The nodetool snapshot command has keyspace and column family options (from nodetool --help):

snapshot [keyspaces...] -cf [columnfamilyName] -t [snapshotName] - Take a snapshot of the optionally specified column family of the specified keyspaces using optional name snapshotName


On Wed, Nov 28, 2012 at = 5:40 AM, Adeel Akbar <adeel.akbar@panasiangroup.com> wrote:
Dear = All,

I have Cassandra 1.1.4 cluster with 2 nodes. I need to take backup and restore on staging for testing purpose. I have taken snapshot with below mentioned command but It created snapshot on every Keyspace's column family. Is there any other way to take backup and restore quick.

/opt/apache-cassandra-1.1.4/bin/nodetool -h localhost snapshot -t cassandra_bkup

Snapshot directory:
= /var/log/cassandra/data/<KeySpace>/<subfolder/snapshot/cassandra_= bkup

--


Thanks & Regards

Adeel = Akbar




--
Tyler Hobbs
DataStax




--
<Mail Attachment.jpeg>
www.groupalia.com
Tom=E0s N=FA=F1ez
IT-Sysprod
Tel. + 34 93 159 31 00 
Fax. + 34 93 396 18 52
Llull, 95-97, 2=BA planta, 08005 Barcelona
Skype: tomas.nunez.groupalia
tomas.nunez@groupalia.com
<Mail = Attachment.png> Twitter    <Mail = Attachment.png> Facebook    <Mail = Attachment.png> Linkedin





= --Apple-Mail=_EF1A1A7A-3FA1-44B1-A028-A72FB68BBE06--