cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-8589) Reconciliation in presence of tombstone might yield state data
Date Mon, 16 Nov 2015 15:51:11 GMT


Sylvain Lebresne commented on CASSANDRA-8589:

I'll note that I've marked 2.1 in the 'fix versions', but it's quite possible the changes
involved to fix this won't be reasonable for 2.1 (after all, this has been there forever and
no-one has reported this from the field).

> Reconciliation in presence of tombstone might yield state data
> --------------------------------------------------------------
>                 Key: CASSANDRA-8589
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Sylvain Lebresne
>             Fix For: 2.1.x, 2.2.x
> Consider 3 replica A, B, C (so RF=3) and consider that we do the following sequence of
actions at {{QUORUM}} where I indicate the replicas acknowledging each operation (and let's
assume that a replica that don't ack is a replica that don't get the update):
> {noformat}
> CREATE TABLE test (k text, t int, v int, PRIMARY KEY (k, t))
> INSERT INTO test(k, t, v) VALUES ('k', 0, 0); // acked by A, B and C
> INSERT INTO test(k, t, v) VALUES ('k', 1, 1); // acked by A, B and C
> INSERT INTO test(k, t, v) VALUES ('k', 2, 2); // acked by A, B and C
> DELETE FROM test WHERE k='k' AND t=1;         // acked by A and C
> UPDATE test SET v = 3 WHERE k='k' AND t=2;    // acked by B and C
> SELECT * FROM test WHERE k='k' LIMIT 2;       // answered by A and B
> {noformat}
> Every operation has achieved quorum, but on the last read, A will respond {{0->0,
tombstone 1, 2->2}} and B will respond {{0->0, 1->1}}. As a consequence we'll answer
{{0->0, 2->2}} which is incorrect (we should respond {{0->0, 2->3}}).
> Put another way, if we have a limit, every replica honors that limit but since tombstones
can "suppress" results from other nodes, we may have some cells for which we actually don't
get a quorum of response (even though we globally have a quorum of replica responses).
> In practice, this probably occurs rather rarely and so the "simpler" fix is probably
to do something similar to the "short reads protection": detect when this could have happen
(based on how replica response are reconciled) and do an additional request in that case.
That detection will have potential false positives but I suspect we can be precise enough
that those false positives will be very very rare (we should nonetheless track how often this
code gets triggered and if we see that it's more often than we think, we could pro-actively
bump user limits internally to reduce those occurrences).

This message was sent by Atlassian JIRA

View raw message