Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Date: Thu, 15 Jul 2010 13:28:06 -0700
From: Anthony Molinaro <anthonym@alumni.caltech.edu>
To: user@cassandra.apache.org
Subject: Re: Bootstrap question
Message-ID: <20100715202806.GB71234@alumni.caltech.edu>
Mail-Followup-To: user@cassandra.apache.org
References: <20100714225847.GA64220@alumni.caltech.edu>
 <AANLkTilt07GBogJ182y-iDz3UcYtT08IXSWUN73Tp6cG@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <AANLkTilt07GBogJ182y-iDz3UcYtT08IXSWUN73Tp6cG@mail.gmail.com>
User-Agent: Mutt/1.4.2.3i

This is a cluster which is horribly imbalanced because I didn't assign
initial tokens, so I'm adding 6 nodes with tokens according to the operations
page (ie, i * (2^127/N) with N = 6).

So here's what the ring will look like when bootstrap finishes

                     151901684708361811491018697633480111658
   Old 1  673.76 GB    1620761242680682425026573496599110901
   Old 2  204.90 GB   10637639655367601517656788464652024082
   Old 3  139.82 GB   21604748163853165203168832909938143241
   New 1              28356863910078205288614550619314017621
   Old 4  250.61 GB   46182405069378676149148922496055212595
   New 2              56713727820156410577229101238628035242
   New 3              85070591730234615865843651857942052863
   Old 5  572.91 GB  103509928471922053310251250943275708086
   New 4             113427455640312821154458202477256070485
   New 5             141784319550391026443072753096570088106
   Old 6  739.61 GB  151901684708361811491018697633480111658
   New 6             170141183460469231731687303715884105728

So from this it seems like I should see anti-compacition on old nodes
4, 5, 6 and 1.

Looking now, it seem that 1 and 6 have had some anti-compaction
happen, node 4 has

 INFO [STREAM-STAGE:1] 2010-07-14 20:53:26,579 StreamOut.java (line 95)
  Performing anticompaction ...

in the log but not a corresponding

CompactionManager.java (line 339) AntiCompacting [..]

line

Node 5 has nothing in its logs about anti-compaction.

Is the fact that 2 new nodes are in the range messing it up?  And if so
how do I recover (I'm thinking, shutdown new nodes 2,3,4,5, the bringing
up nodes 2,4, waiting for them to finish, then bringing up 3,5?).

-Anthony


On Wed, Jul 14, 2010 at 08:45:45PM -0500, Jonathan Ellis wrote:
> Each node logs what token it is going to bootstrap to.  Who owns the
> ranges that contain those tokens?
> 
> On Wed, Jul 14, 2010 at 5:58 PM, Anthony Molinaro
> <anthonym@alumni.caltech.edu> wrote:
> > Hi,
> >
> > �I have a 0.6.3 cluster which contains 6 nodes. �I added 6 new nodes
> > by setting AutoBootstrap to true and setting an InitialToken on each new
> > node, then waiting for the "Bootstrapping" message in the log before
> > starting another. �Then I've been watching the logs on the old boxes
> > waiting to see AntiCompaction messages.
> >
> > Unfortunately after several hours I only see 1 of the 6 old nodes has
> > the AntiCompaction message. �The new nodes are placed such that every
> > old node should have some data pulled from it. �Why don't I see more
> > Anti Compaction messages? �Are there other things I should be looking
> > at?
> >
> > Thanks,
> >
> > -Anthony
> >
> > --
> > ------------------------------------------------------------------------
> > Anthony Molinaro � � � � � � � � � � � � � <anthonym@alumni.caltech.edu>
> >
> 
> 
> 
> -- 
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com

-- 
------------------------------------------------------------------------
Anthony Molinaro                           <anthonym@alumni.caltech.edu>