Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Tue, 29 Aug 2017 11:38:01 +0000 (UTC)
From: "Aleksey Yeschenko (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.13092855.1502104520000.145520.1504006681379@Atlassian.JIRA>
In-Reply-To: <JIRA.13092855.1502104520000@Atlassian.JIRA>
References: <JIRA.13092855.1502104520000@Atlassian.JIRA> <JIRA.13092855.1502104520378@jira-lw-us.apache.org>
Subject: [jira] [Updated] (CASSANDRA-13747) Fix AssertionError in short read
 protection
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Tue, 29 Aug 2017 11:38:08 -0000


     [ https://issues.apache.org/jira/browse/CASSANDRA-13747?page=3Dcom.atl=
assian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aleksey Yeschenko updated CASSANDRA-13747:
------------------------------------------
       Resolution: Fixed
    Fix Version/s:     (was: 3.11.x)
                       (was: 3.0.x)
                   3.11.1
                   3.0.15
           Status: Resolved  (was: Patch Available)

> Fix AssertionError in short read protection
> -------------------------------------------
>
>                 Key: CASSANDRA-13747
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1374=
7
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Coordination
>            Reporter: Aleksey Yeschenko
>            Assignee: Aleksey Yeschenko
>             Fix For: 3.0.15, 3.11.1
>
>
> {{ShortReadRowProtection.moreContents()}} expects that by the time we get=
 to that method, the global post-reconciliation counter was already applied=
 to the current partition. However, sometimes it won=E2=80=99t get applied,=
 and the global counter continues counting with {{rowInCurrentPartition}} v=
alue not reset from previous partition, which in the most obvious case woul=
d trigger the assertion we are observing - {{assert !postReconciliationCoun=
ter.isDoneForPartition();}}. In other cases it=E2=80=99s possible because o=
f this lack of reset to query a node for too few extra rows, causing unnece=
ssary SRP data requests.
> Why is the counter not always applied to the current partition?
> The merged {{PartitionIterator}} returned from {{DataResolver.resolve()}}=
 has two transformations applied to it, in the following order:
> {{Filter}} - to purge non-live data from partitions, and to discard empty=
 partitions altogether (except for Thrift)
> {{Counter}}, to count and stop iteration
> Problem is, {{Filter}} =E2=80=99s {{applyToPartition()}} code that discar=
ds empty partitions ({{closeIfEmpty()}} method) would sometimes consume the=
 iterator, triggering short read protection *before* {{Counter}} =E2=80=99s=
 {{applyToPartition()}} gets called and resets its {{rowInCurrentPartition}=
} sub-counter.
> We should not be consuming iterators until all transformations are applie=
d to them. For transformations it means that they cannot consume iterators =
unless they are the last transformation on the stack.
> The linked branch fixes the problem by splitting {{Filter}} into two tran=
sformations. The original - {{Filter}} - that does filtering within partiti=
ons - and a separate {{EmptyPartitionsDiscarder}}, that discards empty part=
itions from {{PartitionIterators}}. Thus {{DataResolve.resolve()}}, when co=
nstructing its {{PartitionIterator}}, now does merge first, then applies {{=
Filter}}, then {{Counter}}, and only then, as its last (third) transformati=
on - the {{EmptyPartitionsDiscarder}}. Being the last one applied, it=E2=80=
=99s legal for it to consume the iterator, and triggering {{moreContents()}=
} is now no longer a problem.
> Fixes: [3.0|https://github.com/iamaleksey/cassandra/commits/13747-3.0], [=
3.11|https://github.com/iamaleksey/cassandra/commits/13747-3.11], [4.0|http=
s://github.com/iamaleksey/cassandra/commits/13747-4.0]. dtest [here|https:/=
/github.com/iamaleksey/cassandra-dtest/commits/13747].


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org