incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tamar Fraenkel <ta...@tok-media.com>
Subject Re: Upgrade 1.1.2 -> 1.1.6
Date Wed, 21 Nov 2012 06:53:26 GMT
Hi!
I had the same problem (over counting due to replay of commit log, which
ignored drain) after upgrading my cluster from 1.0.9 to 1.0.11.
I updated the Cassandra tickets mentioned in this thread.
Regards,
*Tamar Fraenkel *
Senior Software Engineer, TOK Media

[image: Inline image 1]

tamar@tok-media.com
Tel:   +972 2 6409736
Mob:  +972 54 8356490
Fax:   +972 2 5612956





On Tue, Nov 20, 2012 at 11:03 PM, Mike Heffner <mike@librato.com> wrote:

>
> On Tue, Nov 20, 2012 at 2:49 PM, Rob Coli <rcoli@palominodb.com> wrote:
>
>> On Mon, Nov 19, 2012 at 7:18 PM, Mike Heffner <mike@librato.com> wrote:
>> > We performed a 1.1.3 -> 1.1.6 upgrade and found that all the logs
>> replayed
>> > regardless of the drain.
>>
>> Your experience and desire for different (expected) behavior is welcomed
>> on :
>>
>> https://issues.apache.org/jira/browse/CASSANDRA-4446
>>
>> "nodetool drain sometimes doesn't mark commitlog fully flushed"
>>
>> If every production operator who experiences this issue shares their
>> experience on this bug, perhaps the project will acknowledge and
>> address it.
>>
>>
> Well in this case I think our issue was that upgrading from
> nanotime->epoch seconds, by definition, replays all commit logs. That's not
> due to any specific problem with nodetool drain not marking commitlog's
> flushed, but a safety to ensure data is not lost due to buggy nanotime
> implementations.
>
> For us, it was that the upgrade instructions pre-1.1.5->1.1.6 didn't
> mention that CL's should be removed if successfully drained. On the other
> hand, we do not use counters so replaying them was merely a much longer
> MTT-Return after restarting with 1.1.6.
>
> Mike
>
> --
>
>   Mike Heffner <mike@librato.com>
>   Librato, Inc.
>
>
>

Mime
View raw message