Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of daveviner@gmail.com
 designates 209.85.215.172 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type;
        b=oXnmmX4Oqu04wdE5E2KJovHEFjgdXR3hlYQC0ar/SnOcIeHQPCnuEmRXyq3eBVfv4z
         nUrDgTggn7vRrL1jK+BQ3ulBvIr45fcUCLSdYlvQ0hV74DCSirbYaBU1qwC3wjNw1sEN
         9Nee9C+9IyKf3sOIKi4aj9bbQjXFfZ1dXFOOI=
MIME-Version: 1.0
In-Reply-To: <AANLkTikWsuUOjEU228niWi0iDTAO5J=5wO=i=hG33AnM@mail.gmail.com>
References: <AANLkTikWsuUOjEU228niWi0iDTAO5J=5wO=i=hG33AnM@mail.gmail.com>
Date: Fri, 18 Mar 2011 10:15:38 -0700
Message-ID: <AANLkTinaR_eX4Pch2qDmiNpLtoyqp8=jWYPgOovwbnC4@mail.gmail.com>
Subject: Re: EC2 - 2 regions
From: Dave Viner <daveviner@gmail.com>
To: user@cassandra.apache.org
Cc: A J <s5alye@gmail.com>
Content-Type: multipart/alternative; boundary=002354477df84c2294049ec4ed88

--002354477df84c2294049ec4ed88
Content-Type: text/plain; charset=ISO-8859-1

Hi AJ,

I'd suggest getting to a multi-region cluster step-by-step.  First, get 2
nodes running in the same availability zone.  Make sure that works properly.
 Second, add a node in a separate availability zone, but in the same region.
 Make sure that's working properly.  Third, add a node that's in a separate
region.

Taking it step-by-step will ensure that any issues are specific to the
region-to-region communication, rather than intra-zone connectivity or
cassandra cluster configuration.

Dave Viner


On Fri, Mar 18, 2011 at 8:34 AM, A J <s5alye@gmail.com> wrote:

> Hello,
>
> I am trying to setup a cassandra cluster across regions.
> For testing I am keeping it simple and just having one node in US-EAST
> (say ec2-1-2-3-4.compute-1.amazonaws.com) and one node in US-WEST (say
> ec2-2-2-3-4.us-west-1.compute.amazonaws.com).
> Using Cassandra 0.7.4
>
>
> The one in east region is the seed node and has the values as:
> auto_bootstrap: false
> seeds: ec2-1-2-3-4.compute-1.amazonaws.com
> listen_address: ec2-1-2-3-4.compute-1.amazonaws.com
> rpc_address: 0.0.0.0
>
> The one in west region is non seed and has the values as:
> auto_bootstrap: true
> seeds: ec2-1-2-3-4.compute-1.amazonaws.com
> listen_address: ec2-2-2-3-4.us-west-1.compute.amazonaws.com
> rpc_address: 0.0.0.0
>
> I first fire the seed node (east region instance) and it comes up
> without issues.
> When I fire the non-seed node (west region instance) it fails after
> sometime with the error:
>
> DEBUG 15:09:08,844 Created HHOM instance, registered MBean.
>  INFO 15:09:08,844 Joining: getting load information
>  INFO 15:09:08,845 Sleeping 90000 ms to wait for load information...
> DEBUG 15:09:09,822 attempting to connect to
> ec2-1-2-3-4.compute-1.amazonaws.com/1.2.3.4
> DEBUG 15:09:10,825 Disseminating load info ...
> DEBUG 15:10:10,826 Disseminating load info ...
> DEBUG 15:10:38,845 ... got load info
>  INFO 15:10:38,845 Joining: getting bootstrap token
> ERROR 15:10:38,847 Exception encountered during startup.
> java.lang.RuntimeException: No other nodes seen!  Unable to bootstrap
>        at
> org.apache.cassandra.dht.BootStrapper.getBootstrapSource(BootStrapper.java:164)
>        at
> org.apache.cassandra.dht.BootStrapper.getBalancedToken(BootStrapper.java:146)
>        at
> org.apache.cassandra.dht.BootStrapper.getBootstrapToken(BootStrapper.java:141)
>        at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:450)
>        at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:404)
>        at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:192)
>        at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:314)
>        at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
>
>
> The seed node seems to somewhat acknowledge the non-seed node:
> attempting to connect to /2.2.3.4
> attempting to connect to /10.170.190.31
>
> Can you suggest how can I fix it (I did see a few threads on similar
> issue but did not really follow the chain)
>
> Thanks, AJ
>

--002354477df84c2294049ec4ed88
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi AJ,<div><br></div><div>I&#39;d suggest getting to a multi-region cluster=
 step-by-step. =A0First, get 2 nodes running in the same availability zone.=
 =A0Make sure that works properly. =A0Second, add a node in a separate avai=
lability zone, but in the same region. =A0Make sure that&#39;s working prop=
erly. =A0Third, add a node that&#39;s in a separate region.</div>
<div><br></div><div>Taking it step-by-step will ensure that any issues are =
specific to the region-to-region communication, rather than intra-zone conn=
ectivity or cassandra cluster configuration.</div><div><br></div><div>Dave =
Viner</div>
<div><br><br><div class=3D"gmail_quote">On Fri, Mar 18, 2011 at 8:34 AM, A =
J <span dir=3D"ltr">&lt;<a href=3D"mailto:s5alye@gmail.com">s5alye@gmail.co=
m</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hello,<br>
<br>
I am trying to setup a cassandra cluster across regions.<br>
For testing I am keeping it simple and just having one node in US-EAST<br>
(say <a href=3D"http://ec2-1-2-3-4.compute-1.amazonaws.com" target=3D"_blan=
k">ec2-1-2-3-4.compute-1.amazonaws.com</a>) and one node in US-WEST (say<br=
>
<a href=3D"http://ec2-2-2-3-4.us-west-1.compute.amazonaws.com" target=3D"_b=
lank">ec2-2-2-3-4.us-west-1.compute.amazonaws.com</a>).<br>
Using Cassandra 0.7.4<br>
<br>
<br>
The one in east region is the seed node and has the values as:<br>
auto_bootstrap: false<br>
seeds: <a href=3D"http://ec2-1-2-3-4.compute-1.amazonaws.com" target=3D"_bl=
ank">ec2-1-2-3-4.compute-1.amazonaws.com</a><br>
listen_address: <a href=3D"http://ec2-1-2-3-4.compute-1.amazonaws.com" targ=
et=3D"_blank">ec2-1-2-3-4.compute-1.amazonaws.com</a><br>
rpc_address: 0.0.0.0<br>
<br>
The one in west region is non seed and has the values as:<br>
auto_bootstrap: true<br>
seeds: <a href=3D"http://ec2-1-2-3-4.compute-1.amazonaws.com" target=3D"_bl=
ank">ec2-1-2-3-4.compute-1.amazonaws.com</a><br>
listen_address: <a href=3D"http://ec2-2-2-3-4.us-west-1.compute.amazonaws.c=
om" target=3D"_blank">ec2-2-2-3-4.us-west-1.compute.amazonaws.com</a><br>
rpc_address: 0.0.0.0<br>
<br>
I first fire the seed node (east region instance) and it comes up<br>
without issues.<br>
When I fire the non-seed node (west region instance) it fails after<br>
sometime with the error:<br>
<br>
DEBUG 15:09:08,844 Created HHOM instance, registered MBean.<br>
=A0INFO 15:09:08,844 Joining: getting load information<br>
=A0INFO 15:09:08,845 Sleeping 90000 ms to wait for load information...<br>
DEBUG 15:09:09,822 attempting to connect to<br>
<a href=3D"http://ec2-1-2-3-4.compute-1.amazonaws.com/1.2.3.4" target=3D"_b=
lank">ec2-1-2-3-4.compute-1.amazonaws.com/1.2.3.4</a><br>
DEBUG 15:09:10,825 Disseminating load info ...<br>
DEBUG 15:10:10,826 Disseminating load info ...<br>
DEBUG 15:10:38,845 ... got load info<br>
=A0INFO 15:10:38,845 Joining: getting bootstrap token<br>
ERROR 15:10:38,847 Exception encountered during startup.<br>
java.lang.RuntimeException: No other nodes seen! =A0Unable to bootstrap<br>
 =A0 =A0 =A0 =A0at org.apache.cassandra.dht.BootStrapper.getBootstrapSource=
(BootStrapper.java:164)<br>
 =A0 =A0 =A0 =A0at org.apache.cassandra.dht.BootStrapper.getBalancedToken(B=
ootStrapper.java:146)<br>
 =A0 =A0 =A0 =A0at org.apache.cassandra.dht.BootStrapper.getBootstrapToken(=
BootStrapper.java:141)<br>
 =A0 =A0 =A0 =A0at org.apache.cassandra.service.StorageService.joinTokenRin=
g(StorageService.java:450)<br>
 =A0 =A0 =A0 =A0at org.apache.cassandra.service.StorageService.initServer(S=
torageService.java:404)<br>
 =A0 =A0 =A0 =A0at org.apache.cassandra.service.AbstractCassandraDaemon.set=
up(AbstractCassandraDaemon.java:192)<br>
 =A0 =A0 =A0 =A0at org.apache.cassandra.service.AbstractCassandraDaemon.act=
ivate(AbstractCassandraDaemon.java:314)<br>
 =A0 =A0 =A0 =A0at org.apache.cassandra.thrift.CassandraDaemon.main(Cassand=
raDaemon.java:79)<br>
<br>
<br>
The seed node seems to somewhat acknowledge the non-seed node:<br>
attempting to connect to /<a href=3D"http://2.2.3.4" target=3D"_blank">2.2.=
3.4</a><br>
attempting to connect to /<a href=3D"http://10.170.190.31" target=3D"_blank=
">10.170.190.31</a><br>
<br>
Can you suggest how can I fix it (I did see a few threads on similar<br>
issue but did not really follow the chain)<br>
<br>
Thanks, AJ<br>
</blockquote></div><br></div>

--002354477df84c2294049ec4ed88--