incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Redmumba <redmu...@gmail.com>
Subject Re: Question about replacing a dead node
Date Tue, 03 Jun 2014 18:01:47 GMT
Repairing the range is an expensive operation and don't forget--just
because a node is down does not mean it's dead.  I take nodes down for
maintenance all the time--maybe there was a security update that needed to
be applied, for example, or perhaps a kernel update.  There are a multitude
of reasons why a node would be dead, but not replaced.

If you really, really wanted this to be automated, it'd be trivial to setup
a cron job that looked for dead nodes and removed them from the
cluster--then ran a repair on all of the nodes in your cluster.  This will
cause spikes, especially if you have a large cluster.

Andrew


On Tue, Jun 3, 2014 at 9:45 AM, Ipremyadav <ipremyadav@gmail.com> wrote:

> Thanks Mongo maven:)
> I understand why you need to to do this.
> My question was more from the architecture point if view. Why doesn't
> Cassandra just redistribute the data? Is it because of the gossip protocol?
>
> Thanks,
> Prem
>
> On 3 Jun 2014, at 17:35, Curious Patient <mongomaven@gmail.com> wrote:
>
> Assuming replication factor is >2, if a node dies, why does it matter? If
>> we add a new node is added, shouldn't it just take the chunk of data it
>> server as the "primary" node from the other existing nodes.
>> Why do we need to worry about replacing the dead node?
>
>
> The reason this matters is because I am unable to do a nodetool repair on
> my keyspace with the dead node still being listed in nodetool status.  It
> fails complaining that it cant't reach the dead node.
>
>
> On Tue, Jun 3, 2014 at 12:18 PM, Jeremy Jongsma <jeremy@barchart.com>
> wrote:
>
>> A dead node is still allocated key ranges, and Cassandra will wait for it
>> to come back online rather than redistributing its data. It needs to be
>> decommissioned or replaced by a new node for it to be truly dead as far as
>> the cluster is concerned.
>>
>>
>> On Tue, Jun 3, 2014 at 11:12 AM, Prem Yadav <ipremyadav@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> in the last week week, we saw at least two emails about dead node
>>> replacement. Though I saw the documentation about how to do this, i am not
>>> sure I understand why this is required.
>>>
>>> Assuming replication factor is >2, if a node dies, why does it matter?
>>> If we add a new node is added, shouldn't it just take the chunk of data it
>>> server as the "primary" node from the other existing nodes.
>>> Why do we need to worry about replacing the dead node?
>>>
>>> Thanks
>>>
>>
>>
>

Mime
View raw message