Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 5 Apr 2017 22:32:41 +0000 (UTC)
From: "Tomu Tsuruhara (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13061315.1491271398000.216918.1491431561790@Atlassian.JIRA>
In-Reply-To: <JIRA.13061315.1491271398000@Atlassian.JIRA>
References: <JIRA.13061315.1491271398000@Atlassian.JIRA> <JIRA.13061315.1491271398332@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HBASE-17871) scan#setBatch(int) call leads
 wrong result of VerifyReplication
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Wed, 05 Apr 2017 22:32:46 -0000


    [ https://issues.apache.org/jira/browse/HBASE-17871?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D159=
57920#comment-15957920 ]=20

Tomu Tsuruhara commented on HBASE-17871:
----------------------------------------

@Ted Yu Thanks! Addressed findbugs warning in v3 patch.

"--batch" parsing code was introduced in HBASE-15191.
We don't need to put extra code this time.

> Have you verified the change fixes the inconsistency ?

Yes, I confirmed that in my environment.

before the patch:=20
!=E3=82=B9=E3=82=AF=E3=83=AA=E3=83=BC=E3=83=B3=E3=82=B7=E3=83=A7=E3=83=83=
=E3=83=88 2017-04-06 7.19.53.png|thumbnail=3D1!

after:
!=E3=82=B9=E3=82=AF=E3=83=AA=E3=83=BC=E3=83=B3=E3=82=B7=E3=83=A7=E3=83=83=
=E3=83=88 2017-04-06 7.21.53.png|thumbnail=3D1!

> scan#setBatch(int) call leads wrong result of VerifyReplication
> ---------------------------------------------------------------
>
>                 Key: HBASE-17871
>                 URL: https://issues.apache.org/jira/browse/HBASE-17871
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 1.4.0
>            Reporter: Tomu Tsuruhara
>            Assignee: Tomu Tsuruhara
>            Priority: Minor
>         Attachments: HBASE-17871.master.001.patch, HBASE-17871.master.002=
.patch, HBASE-17871.master.003.patch
>
>
> VerifyReplication tool printed weird logs.
> {noformat}
> 2017-04-03 23:30:50,252 ERROR [main] org.apache.hadoop.hbase.mapreduce.re=
plication.VerifyReplication: CONTENT_DIFFERENT_ROWS, rowkey=3Da000010019300=
00
> 2017-04-03 23:30:50,280 ERROR [main] org.apache.hadoop.hbase.mapreduce.re=
plication.VerifyReplication: ONLY_IN_PEER_TABLE_ROWS, rowkey=3Da00001001930=
000
> 2017-04-03 23:30:50,387 ERROR [main] org.apache.hadoop.hbase.mapreduce.re=
plication.VerifyReplication: CONTENT_DIFFERENT_ROWS, rowkey=3Da000010038500=
00
> 2017-04-03 23:30:50,414 ERROR [main] org.apache.hadoop.hbase.mapreduce.re=
plication.VerifyReplication: ONLY_IN_PEER_TABLE_ROWS, rowkey=3Da00001003850=
000
> 2017-04-03 23:30:50,480 ERROR [main] org.apache.hadoop.hbase.mapreduce.re=
plication.VerifyReplication: CONTENT_DIFFERENT_ROWS, rowkey=3Da000010053200=
00
> 2017-04-03 23:30:50,508 ERROR [main] org.apache.hadoop.hbase.mapreduce.re=
plication.VerifyReplication: ONLY_IN_PEER_TABLE_ROWS, rowkey=3Da00001005320=
000
> {noformat}
> Here, each bad rows were marked as both {{CONTENT_DIFFERENT_ROWS}} and {{=
ONLY_IN_PEER_TABLE_ROWS}}.
> This should never happen so I took a look at code and found scan.setBatch=
 call.
> {code}
>     @Override
>     public void map(ImmutableBytesWritable row, final Result value,
>                     Context context)
>         throws IOException {
>       if (replicatedScanner =3D=3D null) {
> =09    ...
>         final Scan scan =3D new Scan();
>         scan.setBatch(batch);
> {code}
> As stated in HBASE-16376, {{scan#setBatch(int)}} call implicitly allows s=
can results to be partial.
> Since {{VerifyReplication}} is assuming each {{scanner.next()}} call retu=
rns entire row,
> partial results break compare logic.
> We should avoid setBatch call here.
> Thanks to RPC chunking (explained in this blog https://blogs.apache.org/h=
base/entry/scan_improvements_in_hbase_1),
> it's safe and acceptable I think.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)