incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brandon Williams <dri...@gmail.com>
Subject Re: Question on Hinted Handoff
Date Wed, 03 Nov 2010 05:15:51 GMT
On Tue, Nov 2, 2010 at 8:32 AM, Joe Alex <joe.m.alex@gmail.com> wrote:

> Thanks, that clarifies why HH did not work, So have to use .7. Is
> there .6.7 ? I am using .6.6 now.
>

You can use the 0.6 branch instead, it's generally more stable than trunk.


> I did see the log entry in Node 4, adding a Hint for Node 2 and when
> Node 2 came up noticed a log entry in Node 4 that 1 row Handed Off -
> so I thought it was working.
>

This is why it was a pernicious little bug that got me.   Logs indicated it
worked, it just...didn't.

About RR - a read in Node 1 or Node 2 indicated a RR (see log entry
> above) was happening. But dont think that ever happened. Those entries
> keep repeating every time a read in Node 1 and 2 and kept giving old
> data.
>

When you read at CL.ONE, RR is asynchronous -- it happens after your data is
returned.  CL.QUORUM would return the right data after performing the RR.


> AES assuming Anti-Entropy - Tried nodetool repair, this also did not
> fix the issue.
>

That is odd.  Are you sure repair had finished before the test?  In 0.6,
nodetool doesn't block until repair completes (but does in 0.7)


> The only time I saw it being fixed was a read to Node 3 or Node 4
> (mostly Node 3 which acted as responsible when Node 2 was down)  and
> then 2nd reads to Node 2 started showing the latest data.
>

Right, node 3/4 was authoritative for CL.ONE since it had a copy of the
data.  After you read it, RR worked and fixed node 2.


> Do you think RR and AES should have definitely worked ?


I suspect RR worked, and you weren't waiting on AES long enough (it
essentially performs a major compaction, and then streams whatever needs to
be sent to the others, which can take a while depending on the volume of
data.)

-Brandon

Mime
View raw message