Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 66303 invoked from network); 22 Mar 2011 04:29:43 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 22 Mar 2011 04:29:43 -0000 Received: (qmail 84534 invoked by uid 500); 22 Mar 2011 04:29:42 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 84508 invoked by uid 500); 22 Mar 2011 04:29:41 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 84497 invoked by uid 99); 22 Mar 2011 04:29:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Mar 2011 04:29:41 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of milindparikh@gmail.com designates 209.85.214.172 as permitted sender) Received: from [209.85.214.172] (HELO mail-iw0-f172.google.com) (209.85.214.172) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Mar 2011 04:29:34 +0000 Received: by iwl42 with SMTP id 42so8315126iwl.31 for ; Mon, 21 Mar 2011 21:29:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=r+8e1HM5rndWp5vEf5iezQyX9m9Ug83TJ0M1FUGA7gU=; b=AsfXv7WSoPjAEKX+W+xg3fGdhdJDJReNg9WUhnhi9noIuLtisEpR9y7tRUqLXxcDKd kWPbgTG0mmykCZivzAzLa2ZuODvxWq/oy7Ib38Ae0YL0KpbD7vhmKiYaJrsDASr7Q6jQ MxUqi9Oms1+G9hzXN16i6Nq0iW7w0t1NYU9Qg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=rkFa+fKGcRbiQW9nhRPOWTQU78uDLZS36jT9ObVoS1/5lLtjEWFlJ0q/IAF14sQzez CzrJikwhYqEsT/LEcF5RTqyGH7sMB9vpcTAf2JR239K5qMY7YzUVDc9SbSc9Izkusgv+ c6rplmo1XZ7hXQw8aTO9uc3yN02H5hYQJpmX4= MIME-Version: 1.0 Received: by 10.42.97.66 with SMTP id m2mr7941615icn.515.1300768152813; Mon, 21 Mar 2011 21:29:12 -0700 (PDT) Received: by 10.231.149.11 with HTTP; Mon, 21 Mar 2011 21:29:12 -0700 (PDT) In-Reply-To: <8AA667EF-1D7F-4363-A271-783B12015226@gmail.com> References: <2BE7B6CF-F93E-49A5-8C3A-B93766E4B876@gmail.com> <6E63B73B-8205-44D8-99AE-1C5CB2902B63@gmail.com> <8AA667EF-1D7F-4363-A271-783B12015226@gmail.com> Date: Tue, 22 Mar 2011 00:29:12 -0400 Message-ID: Subject: Re: EC2 - 2 regions From: Milind Parikh To: user@cassandra.apache.org Cc: Jeremy Hanna Content-Type: multipart/mixed; boundary=20cf303bfb5eaefffa049f0aaf0f X-Virus-Checked: Checked by ClamAV on apache.org --20cf303bfb5eaefffa049f0aaf0f Content-Type: multipart/alternative; boundary=20cf303bfb5eaefff4049f0aaf0d --20cf303bfb5eaefff4049f0aaf0d Content-Type: text/plain; charset=ISO-8859-1 Patch is attached... I don't have access to Jira. A cautionery note: This is NOT a general solution and is not intended as such. It could be included as a part of larger patch. I will explain in the limitation sections about why it is not a general solution; as I find time. Regards Milind On Mon, Mar 21, 2011 at 11:42 PM, Jeremy Hanna wrote: > Sorry if I was presumptuous earlier. I created a ticket so that the patch > could be submitted and reviewed - that is if it can be generalized so that > it works across regions and doesn't adversely affect the common case. > https://issues.apache.org/jira/browse/CASSANDRA-2362 > > On Mar 21, 2011, at 10:41 PM, Jeremy Hanna wrote: > > > Sorry if I was presumptuous earlier. I created a ticket so that the > patch could be submitted and reviewed - that is if it can be generalized so > that it works across regions and doesn't adversely affect the common case. > > https://issues.apache.org/jira/browse/CASSANDRA-2362 > > > > On Mar 21, 2011, at 12:20 PM, Jeremy Hanna wrote: > > > >> I talked to Matt Dennis in the channel about it and I think everyone > would like to make sure that cassandra works great across multiple regions. > He sounded like he didn't know why it wouldn't work after having looked at > the patches. I would like to try it both ways - with and without the > patches later today if I can and I'd like to help out with getting it > working out of the box. > >> > >> Thanks for the investigative work and documentation Milind! > >> > >> Jeremy > >> > >> On Mar 21, 2011, at 12:12 PM, Dave Viner wrote: > >> > >>> Hi Milind, > >>> > >>> Great work here. Can you provide the patch against the 2 files? > >>> > >>> Perhaps there's some way to incorporate it into the trunk of cassandra > so that this is feasible (in a future release) without patching the source > code. > >>> > >>> Dave Viner > >>> > >>> > >>> On Mon, Mar 21, 2011 at 9:41 AM, A J wrote: > >>> Thanks for sharing the document, Milind ! > >>> Followed the instructions and it worked for me. > >>> > >>> On Mon, Mar 21, 2011 at 5:01 AM, Milind Parikh > wrote: > >>>> Here's the document on Cassandra (0.7.4) across EC2 regions. Clearly > this is > >>>> work in progress.... but wanted to share what I have. PDF is the > working > >>>> copy. > >>>> > >>>> > >>>> > https://docs.google.com/document/d/175duUNIx7m5mCDa2sjXVI04ekyMa5bdiWdu-AFgisaY/edit?hl=en > >>>> > >>>> On Sun, Mar 20, 2011 at 7:49 PM, aaron morton < > aaron@thelastpickle.com> > >>>> wrote: > >>>>> > >>>>> Recent discussion on the dev list > >>>>> http://www.mail-archive.com/dev@cassandra.apache.org/msg01832.html > >>>>> Aaron > >>>>> On 19 Mar 2011, at 06:46, A J wrote: > >>>>> > >>>>> Just to add, all the telnet (port 7000) and cassandra-cli (port 9160) > >>>>> connections are done using the public DNS (that goes like > >>>>> ec2-.....compute.amazonaws.com) > >>>>> > >>>>> On Fri, Mar 18, 2011 at 1:37 PM, A J wrote: > >>>>> > >>>>> I am able to telnet from one region to another on 7000 port without > >>>>> > >>>>> issues. (I get the expected Connected to .....Escape character is > >>>>> > >>>>> '^]'.) > >>>>> > >>>>> Also I am able to execute cassandra client on 9160 port from one > >>>>> > >>>>> region to another without issues (this is when I run cassandra > >>>>> > >>>>> separately on each region without forming a cluster). > >>>>> > >>>>> So I think the ports 7000 and 9160 are not the issue. > >>>>> > >>>>> > >>>>> > >>>>> On Fri, Mar 18, 2011 at 1:26 PM, Dave Viner > wrote: > >>>>> > >>>>> From the us-west instance, are you able to connect to the us-east > instance > >>>>> > >>>>> using telnet on port 7000 and 9160? > >>>>> > >>>>> If not, then you need to open those ports for communication (via your > >>>>> > >>>>> Security Group) > >>>>> > >>>>> Dave Viner > >>>>> > >>>>> On Fri, Mar 18, 2011 at 10:20 AM, A J wrote: > >>>>> > >>>>> Thats exactly what I am doing. > >>>>> > >>>>> I was able to do the first two scenarios without any issues (i.e. 2 > >>>>> > >>>>> nodes in same availability zone. Followed by an additional node in a > >>>>> > >>>>> different zone but same region) > >>>>> > >>>>> I am stuck at the third scenario of separate regions. > >>>>> > >>>>> (I did read the "Cassandra nodes on EC2 in two different regions not > >>>>> > >>>>> communicating" thread but it did not seem to end with resolution) > >>>>> > >>>>> > >>>>> On Fri, Mar 18, 2011 at 1:15 PM, Dave Viner > wrote: > >>>>> > >>>>> Hi AJ, > >>>>> > >>>>> I'd suggest getting to a multi-region cluster step-by-step. First, > get > >>>>> > >>>>> 2 > >>>>> > >>>>> nodes running in the same availability zone. Make sure that works > >>>>> > >>>>> properly. > >>>>> > >>>>> Second, add a node in a separate availability zone, but in the same > >>>>> > >>>>> region. > >>>>> > >>>>> Make sure that's working properly. Third, add a node that's in a > >>>>> > >>>>> separate > >>>>> > >>>>> region. > >>>>> > >>>>> Taking it step-by-step will ensure that any issues are specific to > the > >>>>> > >>>>> region-to-region communication, rather than intra-zone connectivity > or > >>>>> > >>>>> cassandra cluster configuration. > >>>>> > >>>>> Dave Viner > >>>>> > >>>>> On Fri, Mar 18, 2011 at 8:34 AM, A J wrote: > >>>>> > >>>>> Hello, > >>>>> > >>>>> I am trying to setup a cassandra cluster across regions. > >>>>> > >>>>> For testing I am keeping it simple and just having one node in > US-EAST > >>>>> > >>>>> (say ec2-1-2-3-4.compute-1.amazonaws.com) and one node in US-WEST > (say > >>>>> > >>>>> ec2-2-2-3-4.us-west-1.compute.amazonaws.com). > >>>>> > >>>>> Using Cassandra 0.7.4 > >>>>> > >>>>> > >>>>> The one in east region is the seed node and has the values as: > >>>>> > >>>>> auto_bootstrap: false > >>>>> > >>>>> seeds: ec2-1-2-3-4.compute-1.amazonaws.com > >>>>> > >>>>> listen_address: ec2-1-2-3-4.compute-1.amazonaws.com > >>>>> > >>>>> rpc_address: 0.0.0.0 > >>>>> > >>>>> The one in west region is non seed and has the values as: > >>>>> > >>>>> auto_bootstrap: true > >>>>> > >>>>> seeds: ec2-1-2-3-4.compute-1.amazonaws.com > >>>>> > >>>>> listen_address: ec2-2-2-3-4.us-west-1.compute.amazonaws.com > >>>>> > >>>>> rpc_address: 0.0.0.0 > >>>>> > >>>>> I first fire the seed node (east region instance) and it comes up > >>>>> > >>>>> without issues. > >>>>> > >>>>> When I fire the non-seed node (west region instance) it fails after > >>>>> > >>>>> sometime with the error: > >>>>> > >>>>> DEBUG 15:09:08,844 Created HHOM instance, registered MBean. > >>>>> > >>>>> INFO 15:09:08,844 Joining: getting load information > >>>>> > >>>>> INFO 15:09:08,845 Sleeping 90000 ms to wait for load information... > >>>>> > >>>>> DEBUG 15:09:09,822 attempting to connect to > >>>>> > >>>>> ec2-1-2-3-4.compute-1.amazonaws.com/1.2.3.4 > >>>>> > >>>>> DEBUG 15:09:10,825 Disseminating load info ... > >>>>> > >>>>> DEBUG 15:10:10,826 Disseminating load info ... > >>>>> > >>>>> DEBUG 15:10:38,845 ... got load info > >>>>> > >>>>> INFO 15:10:38,845 Joining: getting bootstrap token > >>>>> > >>>>> ERROR 15:10:38,847 Exception encountered during startup. > >>>>> > >>>>> java.lang.RuntimeException: No other nodes seen! Unable to bootstrap > >>>>> > >>>>> at > >>>>> > >>>>> > >>>>> > org.apache.cassandra.dht.BootStrapper.getBootstrapSource(BootStrapper.java:164) > >>>>> > >>>>> at > >>>>> > >>>>> > >>>>> > org.apache.cassandra.dht.BootStrapper.getBalancedToken(BootStrapper.java:146) > >>>>> > >>>>> at > >>>>> > >>>>> > >>>>> > org.apache.cassandra.dht.BootStrapper.getBootstrapToken(BootStrapper.java:141) > >>>>> > >>>>> at > >>>>> > >>>>> > >>>>> > org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:450) > >>>>> > >>>>> at > >>>>> > >>>>> > >>>>> > org.apache.cassandra.service.StorageService.initServer(StorageService.java:404) > >>>>> > >>>>> at > >>>>> > >>>>> > >>>>> > org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:192) > >>>>> > >>>>> at > >>>>> > >>>>> > >>>>> > org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:314) > >>>>> > >>>>> at > >>>>> > >>>>> > org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79) > >>>>> > >>>>> > >>>>> The seed node seems to somewhat acknowledge the non-seed node: > >>>>> > >>>>> attempting to connect to /2.2.3.4 > >>>>> > >>>>> attempting to connect to /10.170.190.31 > >>>>> > >>>>> Can you suggest how can I fix it (I did see a few threads on similar > >>>>> > >>>>> issue but did not really follow the chain) > >>>>> > >>>>> Thanks, AJ > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>> > >>>> > >>> > >> > > > > --20cf303bfb5eaefff4049f0aaf0d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Patch is attached... I don't have access to Jira.
=A0
A cautionery note: This is NOT a general solution and is not intended = as such. It could be included as a part of larger patch. I will explain in = the limitation sections about why it is not a general solution; as I find t= ime.

Regards
Milind
=A0
On Mon, Mar 21, 2011 at 11:42 PM, Jeremy Hanna <= span dir=3D"ltr"><jeremy.h= anna1234@gmail.com> wrote:
Sorry if I was presumptuous earl= ier. =A0I created a ticket so that the patch could be submitted and reviewe= d - that is if it can be generalized so that it works across regions and do= esn't adversely affect the common case.
https://issues.apache.org/jira/browse/CASSANDRA-2362

On = Mar 21, 2011, at 10:41 PM, Jeremy Hanna wrote:

> Sorry if I was p= resumptuous earlier. =A0I created a ticket so that the patch could be submi= tted and reviewed - that is if it can be generalized so that it works acros= s regions and doesn't adversely affect the common case.
> https://issues.apache.org/jira/browse/CASSANDRA-2362
>
> On Mar 21, 2011, at 12:20 PM, Jeremy Hanna w= rote:
>
>> I talked to Matt Dennis in the channel about it a= nd I think everyone would like to make sure that cassandra works great acro= ss multiple regions. =A0He sounded like he didn't know why it wouldn= 9;t work after having looked at the patches. =A0I would like to try it both= ways - with and without the patches later today if I can and I'd like = to help out with getting it working out of the box.
>>
>> Thanks for the investigative work and documentation Mi= lind!
>>
>> Jeremy
>>
>> On Mar 21, 201= 1, at 12:12 PM, Dave Viner wrote:
>>
>>> Hi Milind, >>>
>>> Great work here. =A0Can you provide the patch = against the 2 files?
>>>
>>> Perhaps there's so= me way to incorporate it into the trunk of cassandra so that this is feasib= le (in a future release) without patching the source code.
>>>
>>> Dave Viner
>>>
>>>
= >>> On Mon, Mar 21, 2011 at 9:41 AM, A J <s5alye@gmail.com> wrote:
>>> Thanks for s= haring the document, Milind !
>>> Followed the instructions and it worked for me.
>>>= ;
>>> On Mon, Mar 21, 2011 at 5:01 AM, Milind Parikh <milindparikh@gmail.com> wrote: >>>> Here's the document on Cassandra (0.7.4) across EC2 re= gions. Clearly this is
>>>> work in progress.... but wanted = to share what I have. PDF is the working
>>>> copy.
>&= gt;>>
>>>>
>>>> https://docs.google.com/document/d/175duUNIx7m5mCDa2sjXVI04ekyM= a5bdiWdu-AFgisaY/edit?hl=3Den
>>>>
>>>> On Sun, Mar 20, 2011 at 7:49 PM, aaron= morton <aaron@thelastpickle.= com>
>>>> wrote:
>>>>>
>>&= gt;>> Recent discussion on the dev list
>>>>> http://www.mail-archive.com/dev@= cassandra.apache.org/msg01832.html
>>>>> Aaron
>= ;>>>> On 19 Mar 2011, at 06:46, A J wrote:
>>>>>
>>>>> Just to add, all the telnet (p= ort 7000) and cassandra-cli (port 9160)
>>>>> connections= are done using the public DNS (that goes like
>>>>> ec2-= .....compute.am= azonaws.com)
>>>>>
>>>>> On Fri, Mar 18, 2011 at 1:37 P= M, A J <s5alye@gmail.com> wro= te:
>>>>>
>>>>> I am able to telnet fro= m one region to another on 7000 port without
>>>>>
>>>>> issues. (I get the expected Co= nnected to .....Escape character is
>>>>>
>>>= >> '^]'.)
>>>>>
>>>>> Als= o I am able to execute cassandra client on 9160 port from one
>>>>>
>>>>> region to another without issu= es (this is when I run cassandra
>>>>>
>>>>= ;> separately on each region without forming a cluster).
>>>= >>
>>>>> So I think the ports 7000 and 9160 are not the issue.<= br>>>>>>
>>>>>
>>>>>
= >>>>> On Fri, Mar 18, 2011 at 1:26 PM, Dave Viner <daveviner@gmail.com> wrote:
>>>>>
>>>>> From the us-west instance, are= you able to connect to the us-east instance
>>>>>
>= ;>>>> using telnet on port 7000 and 9160?
>>>>&g= t;
>>>>> If not, then you need to open those ports for communic= ation (via your
>>>>>
>>>>> Security Gr= oup)
>>>>>
>>>>> Dave Viner
>>= >>>
>>>>> On Fri, Mar 18, 2011 at 10:20 AM, A J <s5alye@gmail.com> wrote:
>>>>= >
>>>>> Thats exactly what I am doing.
>>>= >>
>>>>> I was able to do the first two scenarios without any i= ssues (i.e. 2
>>>>>
>>>>> nodes in same= availability zone. Followed by an additional node in a
>>>>= >
>>>>> different zone but same region)
>>>>>= ;
>>>>> I am stuck at the third scenario of separate regi= ons.
>>>>>
>>>>> (I did read the "= Cassandra nodes on EC2 in two different regions not
>>>>>
>>>>> communicating" thread but= it did not seem to end with resolution)
>>>>>
>>= ;>>>
>>>>> On Fri, Mar 18, 2011 at 1:15 PM, Dave= Viner <daveviner@gmail.com&g= t; wrote:
>>>>>
>>>>> Hi AJ,
>>>>>=
>>>>> I'd suggest getting to a multi-region cluster = step-by-step. =A0First, get
>>>>>
>>>>>= 2
>>>>>
>>>>> nodes running in the same avai= lability zone. =A0Make sure that works
>>>>>
>>&= gt;>> properly.
>>>>>
>>>>> Secon= d, add a node in a separate availability zone, but in the same
>>>>>
>>>>> region.
>>>>>= ;
>>>>> Make sure that's working properly. =A0Third, = add a node that's in a
>>>>>
>>>>> = separate
>>>>>
>>>>> region.
>>>>>= ;
>>>>> Taking it step-by-step will ensure that any issue= s are specific to the
>>>>>
>>>>> regio= n-to-region communication, rather than intra-zone connectivity or
>>>>>
>>>>> cassandra cluster configuratio= n.
>>>>>
>>>>> Dave Viner
>>&g= t;>>
>>>>> On Fri, Mar 18, 2011 at 8:34 AM, A J <= ;s5alye@gmail.com> wrote:
>>>>>
>>>>> Hello,
>>>>>=
>>>>> I am trying to setup a cassandra cluster across re= gions.
>>>>>
>>>>> For testing I am kee= ping it simple and just having one node in US-EAST
>>>>>
>>>>> (say ec2-1-2-3-4.compute-1.amaz= onaws.com) and one node in US-WEST (say
>>>>>
>= >>>> ec2-2-2-3-4.us-west-1.compute.amazonaws.com). >>>>>
>>>>> Using Cassandra 0.7.4
>&= gt;>>>
>>>>>
>>>>> The one in = east region is the seed node and has the values as:
>>>>>=
>>>>> auto_bootstrap: false
>>>>>
>&= gt;>>> seeds: ec2-1-2-3-4.compute-1.amazonaws.com
>>&g= t;>>
>>>>> listen_address: ec2-1-2-3-4.compute-1.amazonaws.com=
>>>>>
>>>>> rpc_address: 0.0.0.0
&g= t;>>>>
>>>>> The one in west region is non seed and has the values = as:
>>>>>
>>>>> auto_bootstrap: true>>>>>
>>>>> seeds: ec2-1-2-3-4.compute-1.a= mazonaws.com
>>>>>
>>>>> listen_address: ec2-2-2= -3-4.us-west-1.compute.amazonaws.com
>>>>>
>>= ;>>> rpc_address: 0.0.0.0
>>>>>
>>>>> I first fire the seed node (ea= st region instance) and it comes up
>>>>>
>>>= >> without issues.
>>>>>
>>>>> Wh= en I fire the non-seed node (west region instance) it fails after
>>>>>
>>>>> sometime with the error:
&g= t;>>>>
>>>>> DEBUG 15:09:08,844 Created HHOM = instance, registered MBean.
>>>>>
>>>>>= INFO 15:09:08,844 Joining: getting load information
>>>>>
>>>>> INFO 15:09:08,845 Sleeping 900= 00 ms to wait for load information...
>>>>>
>>&g= t;>> DEBUG 15:09:09,822 attempting to connect to
>>>>&= gt;
>>>>> ec2-1-2-3-4.compute-1.amazonaws.com/1.2.3.4<= br>>>>>>
>>>>> DEBUG 15:09:10,825 Dissemin= ating load info ...
>>>>>
>>>>> DEBUG 15:10:10,826 Disseminati= ng load info ...
>>>>>
>>>>> DEBUG 15:1= 0:38,845 ... got load info
>>>>>
>>>>> = INFO 15:10:38,845 Joining: getting bootstrap token
>>>>>
>>>>> ERROR 15:10:38,847 Exception e= ncountered during startup.
>>>>>
>>>>> = java.lang.RuntimeException: No other nodes seen! =A0Unable to bootstrap
>>>>>
>>>>> =A0 =A0 =A0at
>>>&= gt;>
>>>>>
>>>>> org.apache.cassandr= a.dht.BootStrapper.getBootstrapSource(BootStrapper.java:164)
>>>= ;>>
>>>>> =A0 =A0 =A0at
>>>>>
>>>&= gt;>
>>>>> org.apache.cassandra.dht.BootStrapper.getBa= lancedToken(BootStrapper.java:146)
>>>>>
>>>&= gt;> =A0 =A0 =A0at
>>>>>
>>>>>
>>>>> org.ap= ache.cassandra.dht.BootStrapper.getBootstrapToken(BootStrapper.java:141)>>>>>
>>>>> =A0 =A0 =A0at
>>>= >>
>>>>>
>>>>> org.apache.cassandra.service.S= torageService.joinTokenRing(StorageService.java:450)
>>>>>= ;
>>>>> =A0 =A0 =A0at
>>>>>
>>= >>>
>>>>> org.apache.cassandra.service.StorageService.initServer= (StorageService.java:404)
>>>>>
>>>>> = =A0 =A0 =A0at
>>>>>
>>>>>
>>&g= t;>> org.apache.cassandra.service.AbstractCassandraDaemon.setup(Abstr= actCassandraDaemon.java:192)
>>>>>
>>>>> =A0 =A0 =A0at
>>>&= gt;>
>>>>>
>>>>> org.apache.cassandr= a.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:314= )
>>>>>
>>>>> =A0 =A0 =A0at
>>>&= gt;>
>>>>> org.apache.cassandra.thrift.CassandraDaemon= .main(CassandraDaemon.java:79)
>>>>>
>>>>&= gt;
>>>>> The seed node seems to somewhat acknowledge the non-se= ed node:
>>>>>
>>>>> attempting to conn= ect to /2.2.3.4
>&g= t;>>>
>>>>> attempting to connect to /10.170.190.31
>>>>>
>&g= t;>>> Can you suggest how can I fix it (I did see a few threads on= similar
>>>>>
>>>>> issue but did not really follo= w the chain)
>>>>>
>>>>> Thanks, AJ
= >>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>= ;>>>
>>>>
>>>
>>
>

--20cf303bfb5eaefff4049f0aaf0d-- --20cf303bfb5eaefffa049f0aaf0f Content-Type: application/octet-stream; name="cassec2regions.patch" Content-Disposition: attachment; filename="cassec2regions.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_glkbr2c60 ZGlmZiAtY3JCIHNyYy1vcmlnL2phdmEvb3JnL2FwYWNoZS9jYXNzYW5kcmEvbmV0L01lc3NhZ2lu Z1NlcnZpY2UuamF2YSBzcmMvamF2YS9vcmcvYXBhY2hlL2Nhc3NhbmRyYS9uZXQvTWVzc2FnaW5n U2VydmljZS5qYXZhCioqKiBzcmMtb3JpZy9qYXZhL29yZy9hcGFjaGUvY2Fzc2FuZHJhL25ldC9N ZXNzYWdpbmdTZXJ2aWNlLmphdmEJMjAxMS0wMy0xMSAxOToxNDowMC4wMDAwMDAwMDAgLTA1MDAK LS0tIHNyYy9qYXZhL29yZy9hcGFjaGUvY2Fzc2FuZHJhL25ldC9NZXNzYWdpbmdTZXJ2aWNlLmph dmEJMjAxMS0wMy0yMSAwMDowOToyOS41NDY4NzUwMDAgLTA0MDAKKioqKioqKioqKioqKioqCioq KiAxNjMsMTY5ICoqKioKICAgICAgICAgIFNlcnZlclNvY2tldENoYW5uZWwgc2VydmVyQ2hhbm5l bCA9IFNlcnZlclNvY2tldENoYW5uZWwub3BlbigpOwogICAgICAgICAgZmluYWwgU2VydmVyU29j a2V0IHNzID0gc2VydmVyQ2hhbm5lbC5zb2NrZXQoKTsKICAgICAgICAgIHNzLnNldFJldXNlQWRk cmVzcyh0cnVlKTsKISAgICAgICAgIEluZXRTb2NrZXRBZGRyZXNzIGFkZHJlc3MgPSBuZXcgSW5l dFNvY2tldEFkZHJlc3MobG9jYWxFcCwgRGF0YWJhc2VEZXNjcmlwdG9yLmdldFN0b3JhZ2VQb3J0 KCkpOwogICAgICAgICAgdHJ5CiAgICAgICAgICB7CiAgICAgICAgICAgICAgc3MuYmluZChhZGRy ZXNzKTsKLS0tIDE2MywxNjkgLS0tLQogICAgICAgICAgU2VydmVyU29ja2V0Q2hhbm5lbCBzZXJ2 ZXJDaGFubmVsID0gU2VydmVyU29ja2V0Q2hhbm5lbC5vcGVuKCk7CiAgICAgICAgICBmaW5hbCBT ZXJ2ZXJTb2NrZXQgc3MgPSBzZXJ2ZXJDaGFubmVsLnNvY2tldCgpOwogICAgICAgICAgc3Muc2V0 UmV1c2VBZGRyZXNzKHRydWUpOwohICAgICAgICAgSW5ldFNvY2tldEFkZHJlc3MgYWRkcmVzcyA9 IG5ldyBJbmV0U29ja2V0QWRkcmVzcyhJbmV0QWRkcmVzcy5nZXRMb2NhbEhvc3QoKSwgRGF0YWJh c2VEZXNjcmlwdG9yLmdldFN0b3JhZ2VQb3J0KCkpOwogICAgICAgICAgdHJ5CiAgICAgICAgICB7 CiAgICAgICAgICAgICAgc3MuYmluZChhZGRyZXNzKTsKZGlmZiAtY3JCIHNyYy1vcmlnL2phdmEv b3JnL2FwYWNoZS9jYXNzYW5kcmEvbmV0L091dGJvdW5kVGNwQ29ubmVjdGlvbi5qYXZhIHNyYy9q YXZhL29yZy9hcGFjaGUvY2Fzc2FuZHJhL25ldC9PdXRib3VuZFRjcENvbm5lY3Rpb24uamF2YQoq Kiogc3JjLW9yaWcvamF2YS9vcmcvYXBhY2hlL2Nhc3NhbmRyYS9uZXQvT3V0Ym91bmRUY3BDb25u ZWN0aW9uLmphdmEJMjAxMS0wMy0xMSAxOToxNDowMC4wMDAwMDAwMDAgLTA1MDAKLS0tIHNyYy9q YXZhL29yZy9hcGFjaGUvY2Fzc2FuZHJhL25ldC9PdXRib3VuZFRjcENvbm5lY3Rpb24uamF2YQky MDExLTAzLTIxIDAwOjEzOjUwLjk4NDM3NTAwMCAtMDQwMAoqKioqKioqKioqKioqKioKKioqIDE2 MywxNjkgKioqKgogICAgICAgICAgICAgIHRyeQogICAgICAgICAgICAgIHsKICAgICAgICAgICAg ICAgICAgLy8gemVybyBtZWFucyAnYmluZCBvbiBhbnkgYXZhaWxhYmxlIHBvcnQuJwohICAgICAg ICAgICAgICAgICBzb2NrZXQgPSBuZXcgU29ja2V0KGVuZHBvaW50LCBEYXRhYmFzZURlc2NyaXB0 b3IuZ2V0U3RvcmFnZVBvcnQoKSwgRkJVdGlsaXRpZXMuZ2V0TG9jYWxBZGRyZXNzKCksIDApOwog ICAgICAgICAgICAgICAgICBzb2NrZXQuc2V0S2VlcEFsaXZlKHRydWUpOwogICAgICAgICAgICAg ICAgICBzb2NrZXQuc2V0VGNwTm9EZWxheSh0cnVlKTsKICAgICAgICAgICAgICAgICAgb3V0cHV0 ID0gbmV3IERhdGFPdXRwdXRTdHJlYW0obmV3IEJ1ZmZlcmVkT3V0cHV0U3RyZWFtKHNvY2tldC5n ZXRPdXRwdXRTdHJlYW0oKSwgNDA5NikpOwotLS0gMTYzLDE2OSAtLS0tCiAgICAgICAgICAgICAg dHJ5CiAgICAgICAgICAgICAgewogICAgICAgICAgICAgICAgICAvLyB6ZXJvIG1lYW5zICdiaW5k IG9uIGFueSBhdmFpbGFibGUgcG9ydC4nCiEgICAgICAgICAgICAgICAgIHNvY2tldCA9IG5ldyBT b2NrZXQoZW5kcG9pbnQsIERhdGFiYXNlRGVzY3JpcHRvci5nZXRTdG9yYWdlUG9ydCgpLCBJbmV0 QWRkcmVzcy5nZXRMb2NhbEhvc3QoKSwgMCk7CiAgICAgICAgICAgICAgICAgIHNvY2tldC5zZXRL ZWVwQWxpdmUodHJ1ZSk7CiAgICAgICAgICAgICAgICAgIHNvY2tldC5zZXRUY3BOb0RlbGF5KHRy dWUpOwogICAgICAgICAgICAgICAgICBvdXRwdXQgPSBuZXcgRGF0YU91dHB1dFN0cmVhbShuZXcg QnVmZmVyZWRPdXRwdXRTdHJlYW0oc29ja2V0LmdldE91dHB1dFN0cmVhbSgpLCA0MDk2KSk7Cg== --20cf303bfb5eaefffa049f0aaf0f--