incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Tarbox <tar...@cabotresearch.com>
Subject Re: what addresses to use in EC2 cluster (whenever an instance restarts it gets a new private ip)?
Date Tue, 12 Feb 2013 17:32:18 GMT
When my EC2 instance failed I restarted it, and added the new private IP
address to the list of seed nodes (was this my error?).
Nodetool then showed 4 live nodes and one dead one (corresponding to the
old private IP address).

I'm guessing that what I should have done on the restarted node is start it
with -Dreplace_token?
In such cases what should I do with the list of seed nodes?

I think this is a great opportunity for a technical paper or something on
how to setup Cassandra on EC2.  :-)
BTW: I'm running with encrypted disks....running live on ephemeral drives
that get periodically copied back to EBS stores so I don't lose anything.

Brian


On Tue, Feb 12, 2013 at 12:20 PM, aaron morton <aaron@thelastpickle.com>wrote:

> Cassandra handles nodes changing IP. The import thing to Cassandra is the
> token, not the IP.
>
> In your case did the replacement node have the same token as the failed
> one?
>
> You can normally work around these issues using commands like nodetool
> removetoken.
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> New Zealand
>
> @aaronmorton
> http://www.thelastpickle.com
>
> On 12/02/2013, at 10:04 AM, Andrey Ilinykh <ailinykh@gmail.com> wrote:
>
> You have to use private IPs, but if an instance dies you have to bootstrap
> it with replace token flag. If you use EC2 I'd recommend Netflix's Priam
> tool. It manages all that stuff, plus you have S3 backup.
>
>
> Andrey
>
>
> On Mon, Feb 11, 2013 at 11:35 AM, Brian Tarbox <tarbox@cabotresearch.com>wrote:
>
>> How do I configure my cluster to run in EC2?  In my cassandra.yaml I have
>> IP addresses under seed_provider, listen_address and rpc_address.
>>
>> I tried setting up my cluster using just the EC2 private addresses but
>> when one of my instances failed and I restarted it there was a new private
>> address.  Suddenly my cluster thought it have five nodes rather than four.
>>
>> Then I tried using Elastic IP addresses (permanent addresses) but it
>> turns out you get charged for network traffic between elastic addresses
>> even if they are within the cluster.
>>
>> So...how do you configure the cluster when the IP addresses can change
>> out from under you?
>>
>> Thanks.
>>
>> Brian Tarbox
>>
>
>
>

Mime
View raw message