cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Himanshi Sharma <himanshi.sha...@tcs.com>
Subject Re: Cassandra nodes on EC2 in two different regions not communicating
Date Thu, 24 Feb 2011 12:27:12 GMT
Thanks Daniel.

But SNAT command is not working and when i try tcpdump it gives 

[root@ip-10-136-75-201 ~]# tcpdump -i 50.18.60.117 -n port 7000
tcpdump: Invalid adapter index

Not able to figure out wats this ??

Thanks,
Himanshi




From:
Daniel van Ham Colchete <daniel.colchete@gmail.com>
To:
user@cassandra.apache.org
Date:
02/24/2011 04:27 PM
Subject:
Re: Cassandra nodes on EC2 in two different regions not communicating



Himanshi,

you could try adding your public IP address to an internal interface and 
DNAT the packets to it. This shouldn't give you any problems with your 
normal traffic. Tell Cassandra on listen on the public IPs and it should 
work.

Linux commands would be:

# Create an internal interface using bridge-utils
brctl addbr cassth0

# add the ip
ip addr add dev cassth0 50.18.60.117/32

# DNAT incoming connections
iptables -t nat -A PREROUTING -p tcp --dport 7000 -d INTERNALIP -j DNAT 
--to-destination 50.18.60.117

# SNAT outgoing connections
iptables -t nat -A OUTPUT -p tcp --dport 7000 -d 175.41.143.192 -j SNAT 
--to-source INTERNALIP

This should work since Amazon you re-SNAT your outgoing packets to your 
public IP again, so the other cassandra instance will see your public IP 
as your source address.

I didn't test this setup here but it should work unless I forgot some 
small detail. If you need to troubleshoot use the command "tcpdump -i 
INTERFACE -n port 7000" where INTERFACE should be your public interface or 
your cassth0.

Please let me know if it worked.

Best regards,
Daniel Colchete

On Thu, Feb 24, 2011 at 4:04 AM, Himanshi Sharma <himanshi.sharma@tcs.com> 
wrote:
giving private ip to rpc address gives the same exception
and the keeping it blank and providing public to listen also fails. I 
tried keeping both blank and did telnet on 7000 so i get following o/p
 
[root@ip-10-166-223-150 bin]# telnet 122.248.193.37 7000
Trying 122.248.193.37...
Connected to 122.248.193.37.
Escape character is '^]'.
 
Similarly from another achine
 
[root@ip-10-136-75-201 bin]# telnet 184.72.22.87 7000
Trying 184.72.22.87...
Connected to 184.72.22.87.
Escape character is '^]'.
 


-----Dave Viner wrote: ----- 
To: user@cassandra.apache.org
From: Dave Viner <daveviner@gmail.com>
Date: 02/24/2011 11:59AM
cc: Himanshi Sharma <himanshi.sharma@tcs.com>

Subject: Re: Cassandra nodes on EC2 in two different regions not 
communicating

Try using the private ipv4 address in the rpc_address field, and the 
public ipv4 (NOT the elastic ip) in the listen_address. 

If that fails, go back to rpc_address empty, and start up cassandra. 

Then from the other node, please telnet to port 7000 on the first node. 
 And show the output of that session in your reply. 

I haven't actually constructed a cross-region cluster nor have I used 
v0.7, but this really sounds like it should be easy. 

On Wed, Feb 23, 2011 at 10:22 PM, Himanshi Sharma < 
himanshi.sharma@tcs.com > wrote: 
Hi Dave, 
  
I tried with the public ips. If i mention the public ip in rpc address 
field, Cassandra gives the same exception but if leave it blank then 
Cassandra runs but again in the nodetool command with ring option it 
does'nt show the node in another region. 
  
Thanks, 
Himanshi 


-----Dave Viner wrote: ----- 
To: user@cassandra.apache.org 
From: Dave Viner < daveviner@gmail.com > 
Date: 02/24/2011 10:43AM 

Subject: Re: Cassandra nodes on EC2 in two different regions not 
communicating 

That looks like it's not an issue of communicating between nodes.  It 
appears that the node can not bind to the address on the localhost that 
you're asking for. 

" java.net.BindException: Cannot assign requested address  " 

I think the issue is that the Elastic IP address is not actually an IP 
address that's on the localhost.  So the daemon can not bind to that IP. 
 Instead of using the EIP, use the local IP address for the rpc_address (i 
think that's what you need since that is what Thrift will bind to).  Then 
for the listen_address should be the ip address that is routable from the 
other node.  I would first try with the actual public IP address (not the 
Elastic IP).  Once you get that to work, then shutdown the cluster, change 
the listen_address to the EIP, boot up and try again. 

Dave Viner 


On Wed, Feb 23, 2011 at 8:54 PM, Himanshi Sharma < himanshi.sharma@tcs.com 
> wrote: 

Hey Dave, 

Sorry i forgot to mention the Non-seed configuration. 

for first node in us-west its as below....i.e its own elastic ip 

listen_address: 50.18.60.117 
rpc_address: 50.18.60.117 

and for second node in ap-southeast-1 its as below....i.e again its own 
elastic ip 

listen_address: 175.41.143.192 
rpc_address: 175.41.143.192 

Thanks, 
Himanshi 





From: 
Dave Viner < daveviner@gmail.com > 
To: 
user@cassandra.apache.org 
Date: 
02/23/2011 11:01 PM 
Subject: 
Re: Cassandra nodes on EC2 in two different regions not communicating 




internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g., 
from us-east-1a to us-east-1b) but do not work across regions (e.g., 
us-east to us-west).  To do regions, you must use the public ip address 
assigned by amazon. 

Himanshi, when you log into 1 node, and telnet to port 7000 on the other 
node, which IP address did you use - the 10.x address or the public ip 
address? 
And what is the seed/non-seed configuration in both cassandra.yaml files? 

Dave Viner 


On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio < frank@isidorey.com > 
wrote: 
The internal Amazon IP address is what you will want to use so you don't 
have to go through DNS anyways; not sure if this works from US-East to 
US-West, but it does make things quicker in between zones, e.g. us-east-1a 
to us-east-1b. 


On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner < daveviner@gmail.com > wrote: 

Try using the IP address, not the dns name in the cassandra.yaml. 

If you can telnet from one to the other on port 7000, and both nodes have 
the other node in their config, it should work. 

Dave Viner 


On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma < himanshi.sharma@tcs.com 
> wrote: 

Ya they do. Have specified Public DNS in seed field of each node in 
Cassandra.yaml...nt able to figure out what the problem is ??? 



From: 
Sasha Dolgy < sdolgy@gmail.com > 
To: 
user@cassandra.apache.org 
Date: 
02/23/2011 02:56 PM 
Subject: 
Re: Cassandra nodes on EC2 in two different regions not communicating 





did you define the other host in the cassandra.yaml ?  on both servers 
.... they need to know about each other 

On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma < 
himanshi.sharma@tcs.com > wrote: 

Thanks Dave but I am able to telnet to other instances on port 7000 
and when i run  ./nodetool --host 
ec2-50-18-60-117.us-west-1.compute.amazonaws.com  ring... I can see only 
one node. 

Do we need to configure anything else in Cassandra.yaml or 
Cassandra-env.sh ??? 





From: 
Dave Viner < daveviner@gmail.com > 
To: 
user@cassandra.apache.org 
Cc: 
Himanshi Sharma < himanshi.sharma@tcs.com > 
Date: 
02/23/2011 11:36 AM 
Subject: 
Re: Cassandra nodes on EC2 in two different regions not communicating 






If you login to one of the nodes, can you telnet to port 7000 on the other 
node? 

If not, then almost certainly it's a firewall/Security Group issue. 

You can find out the security groups for any node by logging in, and then 
running: 

% curl " http://169.254.169.254/latest/meta-data/security-groups " 

Assuming that both nodes are in the same security group, ensure that the 
SG is configured to allow other members of the SG to communicate on port 
7000 to each other. 

HTH, 
Dave Viner 


On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma < himanshi.sharma@tcs.com 
> wrote: 

Hi, 

I am new to Cassandra. I m running Cassandra on EC2. I configured 
Cassandra cluster on two instances in different regions. 
But when I am trying the nodetool command with ring option, I am getting 
only single node. 

How to make these two nodes communicate with each other. I have already 
opened required ports. i.e 7000, 8080, 9160 in respective 
security groups. Plz help me with this. 

Regards, 
Himanshi Sharma 


=====-----=====-----===== 
Notice: The information contained in this e-mail 
message and/or attachments to it may contain 
confidential or privileged information. If you are 

not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 

please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you 




=====-----=====-----===== 


Notice: The information contained in this e-mail 
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 


information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 


and any attachments. Thank you 





-- 
Sasha Dolgy 
sasha.dolgy@gmail.com 

=====-----=====-----===== 
Notice: The information contained in this e-mail 
message and/or attachments to it may contain 
confidential or privileged information. If you are 



not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 



please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you 






-- 
Frank LoVecchio 
Senior Software Engineer | Isidorey, LLC 
Google Voice +1.720.295.9179 
isidorey.com | facebook.com/franklovecchio | franklovecchio.com 



=====-----=====-----=====

Notice: The information contained in this e-mail

message and/or attachments to it may contain 

confidential or privileged information. If you are 


not the intended recipient, any dissemination, use, 

review, distribution, printing or copying of the 

information contained in this e-mail message 

and/or attachments to it are strictly prohibited. If 

you have received this communication in error, 


please notify us by reply e-mail or telephone and 

immediately and permanently delete the message 

and any attachments. Thank you




=====-----=====-----=====

Notice: The information contained in this e-mail

message and/or attachments to it may contain 

confidential or privileged information. If you are 


not the intended recipient, any dissemination, use, 

review, distribution, printing or copying of the 

information contained in this e-mail message 

and/or attachments to it are strictly prohibited. If 

you have received this communication in error, 


please notify us by reply e-mail or telephone and 

immediately and permanently delete the message 

and any attachments. Thank you




=====-----=====-----=====

Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 


not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 


please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you




=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you



Mime
View raw message