incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel van Ham Colchete <daniel.colch...@gmail.com>
Subject Re: Cassandra nodes on EC2 in two different regions not communicating
Date Thu, 24 Feb 2011 14:11:59 GMT
Himanshi,

my bad, try this for iptables:

# SNAT outgoing connections
iptables -t nat -A POSTROUTING -p tcp --dport 7000 -d 175.41.143.192 -j SNAT
--to-source INTERNALIP

As for tcpdump the argument for the -i option is the interface name (eth0,
cassth0, etc...), and not the IP. So, it should be
tcpdump -i cassth0 -n port 7000
or
tcpdump -i eth0 -n port 7000

I`m assuming your main network card is eth0, but that should be the case.

Does it work?

Best,
Daniel

On Thu, Feb 24, 2011 at 9:27 AM, Himanshi Sharma <himanshi.sharma@tcs.com>wrote:

>
> Thanks Daniel.
>
> But SNAT command is not working and when i try tcpdump it gives
>
> [root@ip-10-136-75-201 ~]# tcpdump -i 50.18.60.117 -n port 7000
> tcpdump: Invalid adapter index
>
> Not able to figure out wats this ??
>
> Thanks,
> Himanshi
>
>
>
>  From: Daniel van Ham Colchete <daniel.colchete@gmail.com> To:
> user@cassandra.apache.org Date: 02/24/2011 04:27 PM Subject: Re: Cassandra
> nodes on EC2 in two different regions not communicating
> ------------------------------
>
>
>
> Himanshi,
>
> you could try adding your public IP address to an internal interface and
> DNAT the packets to it. This shouldn't give you any problems with your
> normal traffic. Tell Cassandra on listen on the public IPs and it should
> work.
>
> Linux commands would be:
>
> # Create an internal interface using bridge-utils
> brctl addbr cassth0
>
> # add the ip
> ip addr add dev cassth0 *50.18.60.117/32* <http://50.18.60.117/32>
>
> # DNAT incoming connections
> iptables -t nat -A PREROUTING -p tcp --dport 7000 -d INTERNALIP -j DNAT
> --to-destination 50.18.60.117
>
> # SNAT outgoing connections
> iptables -t nat -A OUTPUT -p tcp --dport 7000 -d 175.41.143.192 -j SNAT
> --to-source INTERNALIP
>
> This should work since Amazon you re-SNAT your outgoing packets to your
> public IP again, so the other cassandra instance will see your public IP as
> your source address.
>
> I didn't test this setup here but it should work unless I forgot some small
> detail. If you need to troubleshoot use the command "tcpdump -i INTERFACE -n
> port 7000" where INTERFACE should be your public interface or your cassth0.
>
> Please let me know if it worked.
>
> Best regards,
> Daniel Colchete
>
> On Thu, Feb 24, 2011 at 4:04 AM, Himanshi Sharma <*himanshi.sharma@tcs.com
> * <himanshi.sharma@tcs.com>> wrote:
> giving private ip to rpc address gives the same exception
> and the keeping it blank and providing public to listen also fails. I tried
> keeping both blank and did telnet on 7000 so i get following o/p
>
> [root@ip-10-166-223-150 bin]# telnet 122.248.193.37 7000
> Trying 122.248.193.37...
> Connected to 122.248.193.37.
> Escape character is '^]'.
>
> Similarly from another achine
>
> [root@ip-10-136-75-201 bin]# telnet 184.72.22.87 7000
> Trying 184.72.22.87...
> Connected to 184.72.22.87.
> Escape character is '^]'.
>
>
>
> -----Dave Viner wrote: -----
> To: *user@cassandra.apache.org* <user@cassandra.apache.org>
> From: Dave Viner <*daveviner@gmail.com* <daveviner@gmail.com>>
> Date: 02/24/2011 11:59AM
> cc: Himanshi Sharma <*himanshi.sharma@tcs.com* <himanshi.sharma@tcs.com>>
>
> Subject: Re: Cassandra nodes on EC2 in two different regions not
> communicating
>
> Try using the private ipv4 address in the rpc_address field, and the public
> ipv4 (NOT the elastic ip) in the listen_address.
>
> If that fails, go back to rpc_address empty, and start up cassandra.
>
> Then from the other node, please telnet to port 7000 on the first node.
>  And show the output of that session in your reply.
>
> I haven't actually constructed a cross-region cluster nor have I used v0.7,
> but this really sounds like it should be easy.
>
> On Wed, Feb 23, 2011 at 10:22 PM, Himanshi Sharma < *himanshi.sharma@tcs.com
> * <himanshi.sharma@tcs.com>> wrote:
> Hi Dave,
>
> I tried with the public ips. If i mention the public ip in rpc address
> field, Cassandra gives the same exception but if leave it blank then
> Cassandra runs but again in the nodetool command with ring option it does'nt
> show the node in another region.
>
> Thanks,
> Himanshi
>
>
> -----Dave Viner wrote: -----
> To: *user@cassandra.apache.org * <user@cassandra.apache.org>
> From: Dave Viner < *daveviner@gmail.com * <daveviner@gmail.com>>
> Date: 02/24/2011 10:43AM
>
> Subject: Re: Cassandra nodes on EC2 in two different regions not
> communicating
>
> That looks like it's not an issue of communicating between nodes.  It
> appears that the node can not bind to the address on the localhost that
> you're asking for.
>
> " java.net.BindException: Cannot assign requested address  "
>
> I think the issue is that the Elastic IP address is not actually an IP
> address that's on the localhost.  So the daemon can not bind to that IP.
>  Instead of using the EIP, use the local IP address for the rpc_address (i
> think that's what you need since that is what Thrift will bind to).  Then
> for the listen_address should be the ip address that is routable from the
> other node.  I would first try with the actual public IP address (not the
> Elastic IP).  Once you get that to work, then shutdown the cluster, change
> the listen_address to the EIP, boot up and try again.
>
> Dave Viner
>
>
> On Wed, Feb 23, 2011 at 8:54 PM, Himanshi Sharma < *himanshi.sharma@tcs.com
> * <himanshi.sharma@tcs.com>> wrote:
>
> Hey Dave,
>
> Sorry i forgot to mention the Non-seed configuration.
>
> for first node in us-west its as below....i.e its own elastic ip
>
> listen_address: 50.18.60.117
> rpc_address: 50.18.60.117
>
> and for second node in ap-southeast-1 its as below....i.e again its own
> elastic ip
>
> listen_address: 175.41.143.192
> rpc_address: 175.41.143.192
>
> Thanks,
> Himanshi
>
>
>
>
>   From: Dave Viner < *daveviner@gmail.com * <daveviner@gmail.com>>  To: *user@cassandra.apache.org
> * <user@cassandra.apache.org> Date: 02/23/2011 11:01 PM  Subject: Re:
> Cassandra nodes on EC2 in two different regions not communicating
>
>  ------------------------------
>
>
>
> internal EC2 ips (10.xxx.xxx.xxx) work across availability zones (e.g.,
> from us-east-1a to us-east-1b) but do not work across regions (e.g., us-east
> to us-west).  To do regions, you must use the public ip address assigned by
> amazon.
>
> Himanshi, when you log into 1 node, and telnet to port 7000 on the other
> node, which IP address did you use - the 10.x address or the public ip
> address?
> And what is the seed/non-seed configuration in both cassandra.yaml files?
>
> Dave Viner
>
>
> On Wed, Feb 23, 2011 at 8:12 AM, Frank LoVecchio < *frank@isidorey.com *<frank@isidorey.com>>
> wrote:
> The internal Amazon IP address is what you will want to use so you don't
> have to go through DNS anyways; not sure if this works from US-East to
> US-West, but it does make things quicker in between zones, e.g. us-east-1a
> to us-east-1b.
>
>
> On Wed, Feb 23, 2011 at 9:09 AM, Dave Viner < *daveviner@gmail.com *<daveviner@gmail.com>>
> wrote:
> Try using the IP address, not the dns name in the cassandra.yaml.
>
> If you can telnet from one to the other on port 7000, and both nodes have
> the other node in their config, it should work.
>
> Dave Viner
>
>
> On Wed, Feb 23, 2011 at 1:43 AM, Himanshi Sharma < *himanshi.sharma@tcs.com
> * <himanshi.sharma@tcs.com>> wrote:
>
> Ya they do. Have specified Public DNS in seed field of each node in
> Cassandra.yaml...nt able to figure out what the problem is ???
>
>
>   From: Sasha Dolgy < *sdolgy@gmail.com * <sdolgy@gmail.com>>  To: *user@cassandra.apache.org
> * <user@cassandra.apache.org> Date: 02/23/2011 02:56 PM  Subject: Re:
> Cassandra nodes on EC2 in two different regions not communicating
>
>
>  ------------------------------
>
>
>
> did you define the other host in the cassandra.yaml ?  on both servers ....
> they need to know about each other
>
> On Wed, Feb 23, 2011 at 10:16 AM, Himanshi Sharma < *himanshi.sharma@tcs.com
> * <himanshi.sharma@tcs.com>> wrote:
>
> Thanks Dave but I am able to telnet to other instances on port 7000
> and when i run  ./nodetool --host *ec2-50-18-60-117.us-west-1.compute.amazonaws.com
> * <http://ec2-50-18-60-117.us-west-1.compute.amazonaws.com/> ring... I can
> see only one node.
>
> Do we need to configure anything else in Cassandra.yaml or Cassandra-env.sh
> ???
>
>
>
>
>   From: Dave Viner < *daveviner@gmail.com * <daveviner@gmail.com>>  To: *user@cassandra.apache.org
> * <user@cassandra.apache.org> Cc: Himanshi Sharma < *himanshi.sharma@tcs.com
> * <himanshi.sharma@tcs.com>>  Date: 02/23/2011 11:36 AM  Subject: Re:
> Cassandra nodes on EC2 in two different regions not communicating
>
>
>
>  ------------------------------
>
>
>
> If you login to one of the nodes, can you telnet to port 7000 on the other
> node?
>
> If not, then almost certainly it's a firewall/Security Group issue.
>
> You can find out the security groups for any node by logging in, and then
> running:
>
> % curl " *http://169.254.169.254/latest/meta-data/security-groups *<http://169.254.169.254/latest/meta-data/security-groups>"
>
>
> Assuming that both nodes are in the same security group, ensure that the SG
> is configured to allow other members of the SG to communicate on port 7000
> to each other.
>
> HTH,
> Dave Viner
>
>
> On Tue, Feb 22, 2011 at 8:59 PM, Himanshi Sharma < *himanshi.sharma@tcs.com
> * <himanshi.sharma@tcs.com>> wrote:
>
> Hi,
>
> I am new to Cassandra. I m running Cassandra on EC2. I configured Cassandra
> cluster on two instances in different regions.
> But when I am trying the nodetool command with ring option, I am getting
> only single node.
>
> How to make these two nodes communicate with each other. I have already
> opened required ports. i.e 7000, 8080, 9160 in respective
> security groups. Plz help me with this.
>
> Regards,
> Himanshi Sharma
>
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
>
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
>
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>
>
> =====-----=====-----=====
>
>
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
>
>
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
>
>
> and any attachments. Thank you
>
>
>
>
>
> --
> Sasha Dolgy *
> **sasha.dolgy@gmail.com * <sasha.dolgy@gmail.com>
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
>
>
>
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
>
>
>
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>
>
>
>
> --
> Frank LoVecchio
> Senior Software Engineer | Isidorey, LLC
> Google Voice +1.720.295.9179 *
> **isidorey.com * <http://isidorey.com/>| *facebook.com/franklovecchio *<http://facebook.com/franklovecchio>|
> *franklovecchio.com * <http://franklovecchio.com/>
>
>
>
> =====-----=====-----=====
>
> Notice: The information contained in this e-mail
>
> message and/or attachments to it may contain
>
> confidential or privileged information. If you are
>
>
> not the intended recipient, any dissemination, use,
>
> review, distribution, printing or copying of the
>
> information contained in this e-mail message
>
> and/or attachments to it are strictly prohibited. If
>
> you have received this communication in error,
>
>
> please notify us by reply e-mail or telephone and
>
> immediately and permanently delete the message
>
> and any attachments. Thank you
>
>
>
>
> =====-----=====-----=====
>
> Notice: The information contained in this e-mail
>
> message and/or attachments to it may contain
>
> confidential or privileged information. If you are
>
>
> not the intended recipient, any dissemination, use,
>
> review, distribution, printing or copying of the
>
> information contained in this e-mail message
>
> and/or attachments to it are strictly prohibited. If
>
> you have received this communication in error,
>
>
> please notify us by reply e-mail or telephone and
>
> immediately and permanently delete the message
>
> and any attachments. Thank you
>
>
>
>
> =====-----=====-----=====
>
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
>
>
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
>
>
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>
>
> =====-----=====-----=====
>
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
>
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
>
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>

Mime
View raw message