From user-return-37276-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Mon Oct 28 07:20:01 2013 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3A8D310A76 for ; Mon, 28 Oct 2013 07:20:01 +0000 (UTC) Received: (qmail 29750 invoked by uid 500); 28 Oct 2013 07:19:58 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 29586 invoked by uid 500); 28 Oct 2013 07:19:57 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 29578 invoked by uid 99); 28 Oct 2013 07:19:57 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Oct 2013 07:19:57 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy includes SPF record at spf.trusted-forwarder.org) Received: from [209.85.192.169] (HELO mail-pd0-f169.google.com) (209.85.192.169) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Oct 2013 07:19:51 +0000 Received: by mail-pd0-f169.google.com with SMTP id q10so6667661pdj.14 for ; Mon, 28 Oct 2013 00:19:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-type:message-id:mime-version :subject:date:references:to:in-reply-to; bh=UvbvFUY0arG1lh7Ma9ypUTIWD5Xd8ZXb+a+2hz6uSek=; b=j7a+LPKKkriz16OVsxlnEMk2tLdqk2hEBk/76I8jAdvQkHv2bkOZ4g9djECfnMgPQY 9YP+87B7J19cOFSKhZo6AWcpoYfEs/dcXoA2hOlYkZRIQD71KEchUNTuyVbwL4tmYc5t sT/xMUmnbdC/ZeYevEs8dxz6k2rAmL9VI96fF1K7CI5DM0gKMMjL3wauHjzfwd2j2YU/ DgkJx7KlbonRreMaA6v50+LW1oc3rZFRpjN3NYjuqM+W6k5Plv+nvl1uEu7KvaJJ545z 4LWdikuFyascHBrEwMNNkp3Xd+rtImNfsrUs/T5poWouodSWzfb33GXrg+6x8+xEuU9l kpng== X-Gm-Message-State: ALoCoQmn8lPWa2e4b7vk9rxL2j3iyDAnPhYU49ZQ2Sbb6UJ11kF6SvnnwcOglo2uGCarF0TbD7b/ X-Received: by 10.68.225.197 with SMTP id rm5mr1243564pbc.166.1382944769774; Mon, 28 Oct 2013 00:19:29 -0700 (PDT) Received: from [172.16.1.20] ([203.86.207.101]) by mx.google.com with ESMTPSA id pl1sm8073912pbb.20.2013.10.28.00.19.27 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 28 Oct 2013 00:19:29 -0700 (PDT) From: Aaron Morton Content-Type: multipart/alternative; boundary="Apple-Mail=_0999BF41-907F-4A37-9933-4DD430665A00" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1816\)) Subject: Re: Adding a data center with data already in place Date: Mon, 28 Oct 2013 20:19:25 +1300 References: To: Cassandra User In-Reply-To: X-Mailer: Apple Mail (2.1816) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_0999BF41-907F-4A37-9933-4DD430665A00 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 > Today I need to bring that data center back in. It is not 2-3 days out = dated. I have two options: >=20 > 1) Treat this as a new data center and let the nodes sync from = scratch, or > 2) Bring the nodes back up with all the data in place and do a repair. As long as the nodes were down for less than gc_grace_seconds i would = bring the old ones back with their data and run repair.=20 If possible avoid having the application read from them until it=92s = complete.=20 > There are 4 nodes in both data centers, with RF=3D2. I=92d recommend moving to RF 3 if you use QUORUM.=20 Cheers ----------------- Aaron Morton New Zealand @aaronmorton Co-Founder & Principal Consultant Apache Cassandra Consulting http://www.thelastpickle.com On 26/10/2013, at 2:33 am, Oleg Dulin wrote: > I am using Cassandra 1.1.11 and plan on upgrading soon, but in the = meantime here is what happened. >=20 > I couldn't run repairs because of a slow WAN pipe, so i removed the = second data center from the cluster. >=20 > Today I need to bring that data center back in. It is not 2-3 days out = dated. I have two options: >=20 > 1) Treat this as a new data center and let the nodes sync from = scratch, or > 2) Bring the nodes back up with all the data in place and do a repair. >=20 > We are talking about 30-40Gigs per node. There are 4 nodes in both = data centers, with RF=3D2. >=20 >=20 > --=20 > Regards, > Oleg Dulin > http://www.olegdulin.com >=20 >=20 --Apple-Mail=_0999BF41-907F-4A37-9933-4DD430665A00 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=windows-1252
Today I need to bring that = data center back in. It is not 2-3 days out dated. I have two = options:

1) Treat this as a new data center and let the nodes = sync from scratch, or
2) Bring the nodes back up with all the data in = place and do a repair.
As long as the nodes were down for = less than gc_grace_seconds i would bring the old ones back with their = data and run repair. 

If possible avoid having = the application read from them until it=92s = complete. 

There = are 4 nodes in both data centers, with RF=3D2.
I=92d = recommend moving to RF 3 if you use = QUORUM. 

Cheers

http://www.thelastpickle.com

On 26/10/2013, at 2:33 am, Oleg Dulin <oleg.dulin@gmail.com> = wrote:

I am using Cassandra 1.1.11 and plan on upgrading soon, = but in the meantime here is what happened.

I couldn't run repairs = because of a slow WAN pipe, so i removed the second data center from the = cluster.

Today I need to bring that data center back in. It is = not 2-3 days out dated. I have two options:

1) Treat this as a = new data center and let the nodes sync from scratch, or
2) Bring the = nodes back up with all the data in place and do a repair.

We are = talking about 30-40Gigs per node. There are 4 nodes in both data = centers, with RF=3D2.


--
Regards,
Oleg Dulin
http://www.olegdulin.com


=

= --Apple-Mail=_0999BF41-907F-4A37-9933-4DD430665A00--