cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sameer Farooqui <cassandral...@gmail.com>
Subject Re: Running Cassandra across different Amazon EC2 regions
Date Fri, 06 May 2011 22:24:36 GMT
Thanks, Shasha.

It looks like there are two ways to make Cassandra work across regions,
changing the source code or using software like Vyatta and VIPs.


++ Changing the source code ++
Milind documents how to change the source code in 0.7.4 but warns that there
are significant limitations to this, which are not really discussed in his
paper. I think some people ran into a problem running nodetool repair after
this change, but it worked fine for others. Also, people worried about
inter-node communication not being encrypted last month, but with the
release of 0.8 this isn't a problem. Are there any other problems we can run
into by changing the source code this way?

How to change source code in 0.7.4:
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/attachment/6191521/0/CassandraInstancesindifferentregions--EC2%5B1%5D.pdf?by-user=t


+++ Configuring VIPs +++
Shasha documented a method of using Vyatta (vyatta.com) and VIPs to
configure Cassandra across regions. This technique configures a VIP
interface (tunXXX) for each node and sets the listen address as the VIP, so
all inter-node communication occurs over the VPN tunnel. The RPC address is
set to 0.0.0.0 so it listens on all interfaces and does not use the VPN
tunnel.

- This approach seems to be better, but does anyone have instructions for
doing this in EC2 without Vyatta, which is paid software and costs around
$700? We're using Ubuntu 11.04 on EC2 and would prefer to use an open
source, free software vendor.

- Sameer



On Thu, May 5, 2011 at 10:48 PM, Sasha Dolgy <sdolgy@gmail.com> wrote:

>
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Re-IP-address-resolution-in-MultiDC-setup-EC2-VIP-td6306635.html
> On May 6, 2011 3:07 AM, "Sameer Farooqui" <cassandralabs@gmail.com> wrote:
> > We're trying to set up a Cassandra 0.8.0beta1 cluster across Amazon East
> &
> > West regions. It does not work out of the box with the binaries and the
> > nodes in different regions end up setting this own clusters.
> >
> > The problem is with Cassandra's Listening Address as described by Rui:
> > "Using external IP causes Binding exception, because Cassandra, and java
> > socket, cannot listen on the external IP, which does not belong to this
> > node. Using private IP prevents nodes in other region from connecting
> back
> > to this node with the private IP, which has been sent to other nodes for
> > cluster establishing."
> >
> > In Feb 2011, Peter Fales posted: "... the trick is that you have must use
> > the "external" addresses in your storage-conf.xml, but since you don't
> have
> > a NIC that can actually bind to those addresses, you need to listen on
> the
> > "internal" addresses (or, more simply, all interfaces). At the time, I
> was
> > not able to get the cross-region cluster to work without making changes
> to
> > the Cassandra code. Perhaps things have evolved so that there are other
> ways
> > to do it now."
> >
> > Does anyone know if Cassandra 0.8 can be configured to run in multiple
> > Amazon regions without modifying the source code? And if not, has anybody
> > successfully done this with the 0.8 source code and shared the technique?
> >
> >
> > After some research we found that other people have also ran into this
> issue
> > with versions 0.6.x and 0.7.x of Cassandra:
> >
> > Fix for 0.6.5 by Peter Fales in Sept 2010:
> > http://www.mail-archive.com/dev@cassandra.apache.org/msg01832.html
> > http://www.mail-archive.com/user@cassandra.apache.org/msg05692.html
> >
> >
> > Fix for 0.7 by Rui:
> > http://rliu09.wordpress.com/2011/04/23/cassandra-across-ec2-regions/
> >
> >
> > - Sameer
> > Accenture Technology Labs
>

Mime
View raw message