From user-return-38809-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Wed Feb 5 19:01:31 2014 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6513C10F4F for ; Wed, 5 Feb 2014 19:01:31 +0000 (UTC) Received: (qmail 71922 invoked by uid 500); 5 Feb 2014 19:01:27 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 71896 invoked by uid 500); 5 Feb 2014 19:01:27 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 71888 invoked by uid 99); 5 Feb 2014 19:01:27 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Feb 2014 19:01:27 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,UNPARSEABLE_RELAY X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of kwright@nanigans.com designates 216.82.254.106 as permitted sender) Received: from [216.82.254.106] (HELO mail1.bemta7.messagelabs.com) (216.82.254.106) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Feb 2014 19:01:17 +0000 Received: from [216.82.253.99:54118] by server-10.bemta-7.messagelabs.com id 4C/56-15119-76A82F25; Wed, 05 Feb 2014 19:00:55 +0000 X-Env-Sender: kwright@nanigans.com X-Msg-Ref: server-9.tower-160.messagelabs.com!1391626841!12878450!21 X-Originating-IP: [216.166.12.178] X-StarScan-Received: X-StarScan-Version: 6.9.16; banners=-,-,- X-VirusChecked: Checked Received: (qmail 17584 invoked from network); 5 Feb 2014 19:00:55 -0000 Received: from out001.collaborationhost.net (HELO out001.collaborationhost.net) (216.166.12.178) by server-9.tower-160.messagelabs.com with RC4-SHA encrypted SMTP; 5 Feb 2014 19:00:55 -0000 Received: from AUSP01VMBX28.collaborationhost.net ([192.168.20.73]) by AUSP01MHUB04.collaborationhost.net ([10.2.0.189]) with mapi; Wed, 5 Feb 2014 13:01:54 -0600 From: Keith Wright To: "user@cassandra.apache.org" CC: Don Jackson , Dave Carroll Date: Wed, 5 Feb 2014 13:00:47 -0600 Subject: Move to smaller nodes Thread-Topic: Move to smaller nodes Thread-Index: Ac8ipLviJHusH8ZXRh2Qsxvv9u0+KQ== Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.2.3.120616 acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_CF17F48E1F721kwrightnaniganscom_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_CF17F48E1F721kwrightnaniganscom_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi all, Earlier today I emailed about issues we=92re having bootstrapping nodes= into our existing cluster. One theory we have is that our nodes are simpl= y too large and are considering moving to more, smaller nodes. However, be= cause we cannot bootstrap it makes it difficult. As I see it, we have two = options (assuming the new cluster is already setup and running): * Add the new cluster as another data center. I am already using Networ= kTopologySnitch. The existing nodes would then stream their data over to t= he new cluster. Couple questions here: * I assume its ok if data centers have different node sizes (I.e. Sma= ller) and more nodes? * Is adding a new data center to a cluster basically a large bootstra= p in which case its quite possible our existing bootstrap issues would pres= ent themselves? Documentation via nodetool rebuild indicates it is. * Use SSTableLoader to bulk load data on the existing cluster to the new= one. To do, I would need to do the following steps: * Have clients start dual writes to new and old cluster (only read fr= om old) * Backup data on the nodes. We are using JNA so this should not resu= lt in double the data space usage, correct? I assume I can then simply ftp= the hard links to another server? * Run SSTableLoader on each of the SSTables taken from the backup to = the new cluster * When SSTableLoader has completed, new cluster will have all of the = data and old cluster can be decommissioned Thoughts? Any automated tools around the SSTableLoader option? Thanks --_000_CF17F48E1F721kwrightnaniganscom_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Hi all,

    Earlier today I emailed about issues we=92re having bootstra= pping nodes into our existing cluster.  One theory we have is that our= nodes are simply too large and are considering moving to more, smaller nod= es.  However, because we cannot bootstrap it makes it difficult.  = ;As I see it, we have two options (assuming the new cluster is already setu= p and running):
  • Add the new cluster as another data center. &n= bsp;I am already using NetworkTopologySnitch.  The existing nodes woul= d then stream their data over to the new cluster.  Couple questions he= re:
    • I assume its ok if data centers have different node sizes (= I.e. Smaller) and more nodes?
    • Is adding a new data center to a clus= ter basically a large bootstrap in which case its quite possible our existi= ng bootstrap issues would present themselves?  Documentation via nodet= ool rebuild indicates it is.
  • Use SSTableLoader to bulk load da= ta on the existing cluster to the new one.  To do, I would need to do = the following steps:
    • Have clients start dual writes to new and = old cluster (only read from old)
    • Backup data on the nodes.  We= are using JNA so this should not result in double the data space usage, co= rrect?  I assume I can then simply ftp the hard links to another serve= r?
    • Run SSTableLoader on each of the SSTables taken from the backup = to the new cluster
    • When SSTableLoader has completed, new cluster wi= ll have all of the data and old cluster can be decommissioned
    Thoughts?  Any automated tools around the SSTableLoader option?<= /div>

    Thanks
    --_000_CF17F48E1F721kwrightnaniganscom_--