cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain RODRIGUEZ <arodr...@gmail.com>
Subject Re: Issue when node goes away?
Date Wed, 01 Jul 2015 08:49:05 GMT
Basically, when you add nodes, add them on the correct version to avoid
schema / network issues in your streams. Also, try to update all the node
using rolling restarts in a reduced time frame after stopping repairs, with
all the nodes up, etc.

You must have a healthy cluster before performing an upgrade.

Remember running a nodetool upgradesstable after any upgrade (if not needed
it will end fast, so I would run it always as a best practice, just in
case).

C*heers,

Alain

2015-07-01 2:16 GMT+02:00 David Aronchick <aronchick@gmail.com>:

> That is a GREAT lead! So it looks like I can't add a few nodes to the
> cluster of the new version, have it settle down, and then upgrade the rest?
>
> On Tue, Jun 30, 2015 at 11:58 AM, Alain RODRIGUEZ <arodrime@gmail.com>
> wrote:
>
>> "Would it matter that I'm mixing cassandra versions? "
>>
>> From:
>> http://docs.datastax.com/en/upgrade/doc/upgrade/datastax_enterprise/upgrdLim.html
>>
>> "General upgrade limitations¶
>>
>> Do not run nodetool repair.
>> Do not enable new features.
>> Do not issue these types of queries during a rolling restart: DDL,
>> TRUNCATE
>> *During upgrades, the nodes on different versions show a schema
>> disagreement*."
>>
>> I think this is a good lead.
>>
>> C*heers,
>>
>> Alain
>>
>> 2015-06-30 20:22 GMT+02:00 David Aronchick <aronchick@gmail.com>:
>>
>>> I appreciate the thoughts! My issue is that it seems to work perfectly,
>>> until the node goes away. Would it matter that I'm mixing cassandra
>>> versions? (2.1.4 and 2.1.5)?
>>>
>>> On Tue, Jun 30, 2015 at 5:23 AM, Alain RODRIGUEZ <arodrime@gmail.com>
>>> wrote:
>>>
>>>> Hi David ?
>>>>
>>>> What does a "nodetool describecluster" output look like ?
>>>>
>>>> My guess is you might be having a schema version desynchronisation. If
>>>> you see a node with different schema version you might want to try a
>>>> "nodetool resetlocal*schema*       - Reset node's local *schema* and
>>>> resync"
>>>>
>>>> You asked for any thoughts, this is a thought, not sure if it will
>>>> help, I hope so.
>>>>
>>>> C*heers,
>>>>
>>>> Alain
>>>>
>>>> 2015-06-30 1:44 GMT+02:00 Robert Coli <rcoli@eventbrite.com>:
>>>>
>>>>> On Mon, Jun 29, 2015 at 2:43 PM, David Aronchick <aronchick@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Ping--- any thoughts here?
>>>>>>
>>>>>
>>>>> I don't have any thoughts on your specific issue at this time, but
>>>>> FWIW #cassandra on freenode is sometimes a better forum for interactive
>>>>> debugging of operational edge cases.
>>>>>
>>>>> =Rob
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message