Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Date: Wed, 21 Apr 2010 09:31:56 -0700
From: Anthony Molinaro <anthonym@alumni.caltech.edu>
To: user@cassandra.apache.org
Subject: Re: Clarification on Ring operations in Cassandra 0.5.1
Message-ID: <20100421163156.GB39306@alumni.caltech.edu>
Mail-Followup-To: user@cassandra.apache.org
References: <20100415231044.GA8977@alumni.caltech.edu>
 <s2qe06563881004191328q47743f30l5f0e7293bf5f3eba@mail.gmail.com>
 <20100419234823.GC30486@alumni.caltech.edu>
 <20100421160244.GA39306@alumni.caltech.edu>
 <y2re06563881004210908m3bdfbfe4wcc76ab0b935b5b0e@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <y2re06563881004210908m3bdfbfe4wcc76ab0b935b5b0e@mail.gmail.com>
User-Agent: Mutt/1.4.2.3i


On Wed, Apr 21, 2010 at 11:08:19AM -0500, Jonathan Ellis wrote:
> Yes, that looks right, where "token really close" means "slightly less
> than" (more than would move it into a different node's range).

Is it better to go slightly less than (say Token - 1), or slightly more than
the beginning of the range (PreviousTokenInRing + 1).  I was assuming the
latter in my earlier email, but you seem to be suggesting the former?

> You can't really migrate via scp since only one node with a given
> token can exist in the cluster at a time.

Right, I was mostly wondering if I could speed things up by scping the
sstables while the system was running (since they shouldn't be changing).
Then in quick succession removetoken and bootstrap with the old token.
Probably grasping at straws here :b

Thanks for the answers,

-Anthony

> On Wed, Apr 21, 2010 at 11:02 AM, Anthony Molinaro
> <anthonym@alumni.caltech.edu> wrote:
> > Hi,
> >
> > �I'm still curious if I got the data movement right in this email from
> > before? �Anyone? �Also, anyone know if I can scp the data directory from
> > a node I want to replace to a new machine? �The cassandra streaming seems
> > much slower than scp.
> >
> > -Anthony
> >
> > On Mon, Apr 19, 2010 at 04:48:23PM -0700, Anthony Molinaro wrote:
> >>
> >> On Mon, Apr 19, 2010 at 03:28:26PM -0500, Jonathan Ellis wrote:
> >> > > Can I then 'nodeprobe move <token for range I want to take over>', and
> >> > > achieve the same as step 2 above?
> >> >
> >> > You can't have two nodes with the same token in the ring at once. �So,
> >> > you can removetoken the old node first, then bootstrap the new one
> >> > (just specify InitialToken in the config to avoid having it guess
> >> > one), or you can make it a 3 step process (bootstrap, remove, move) to
> >> > avoid transferring so much data around.
> >>
> >> So I'm still a little fuzzy for your 3 step case on why less data moves,
> >> but let me run through the two scenarios and see where we get. �Please
> >> correct me if I'm wrong on some point.
> >>
> >> Let say I have 3 nodes with random partitioner and rack unaware strategy.
> >> Which means I have something like
> >>
> >> Node �Size � Token �KeyRange (self + next in ring)
> >> ---- �---- � ----- �------------------------------
> >> A � � 5 G � � �33 � �1 -> 66
> >> B � � 6 G � � �66 � � � 34 -> 0
> >> C � � 2 G � � � 0 � � � � �67 -> 33
> >>
> >> Now lets say Node B is giving us some problems, so we want to replace it
> >> with another node D.
> >>
> >> We've outlined 2 processes.
> >>
> >> In the first process you recommend
> >>
> >> 1. removetoken on node B
> >> 2. wait for data to move
> >> 3. add InitialToken of 66 and AutoBootstrap = true to node D storage-conf.xml
> >> � �then start it
> >> 4. wait for data to move
> >>
> >> So when you do the removetoken, this will cause the following transfers
> >> at stage 2
> >> � Node A sends 34->66 to Node C
> >> � Node C sends 67->0 �to Node A
> >> at stage 4
> >> � Node A sends 34->66 to Node D
> >> � Node C sends 67->0 �to Node D
> >>
> >> In the second process I assume you pick a token really close to another token?
> >>
> >> 1. add InitialToken of 34 and AutoBootstrap to true to node D storage-conf.xml
> >> � �then start it
> >> 2. wait for data to move
> >> 3. removetoken on node B
> >> 4. wait for data to move
> >> 5. movetoken on node D to 66
> >> 6. wait for data to move
> >>
> >> This results in the following moves
> >> at stage 2
> >> � Node A/B sends 33->34 to Node D (primary token range)
> >> � Node B sends 34->66 to Node D � (replica range)
> >> at stage 4
> >> � Node C sends 66->0 to Node D (replica range)
> >> at stage 6
> >> � No data movement as D already had 33->0
> >>
> >> So seems like you move all the data twice for process 1 and only a small
> >> portion twice for process 2 (which is what you said, so hopefully I've
> >> outlined correctly what is happening). �Does all that sound right?
> >>
> >> Once I've run bootstrap with the InitialToken value set in the config is
> >> it then ignored in subsequent restarts, and if so can I just remove it
> >> after that first time?
> >>
> >> Thanks,
> >>
> >> -Anthony
> >>
> >> --
> >> ------------------------------------------------------------------------
> >> Anthony Molinaro � � � � � � � � � � � � � <anthonym@alumni.caltech.edu>
> >
> > --
> > ------------------------------------------------------------------------
> > Anthony Molinaro � � � � � � � � � � � � � <anthonym@alumni.caltech.edu>
> >

-- 
------------------------------------------------------------------------
Anthony Molinaro                           <anthonym@alumni.caltech.edu>