hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11487) ScanResponse carries non-zero cellblock for CloseScanRequest (ScanRequest with close_scanner = true)
Date Thu, 10 Jul 2014 03:13:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057077#comment-14057077
] 

Hadoop QA commented on HBASE-11487:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12654910/HBase_11487_v2.patch
  against trunk revision .
  ATTACHMENT ID: 12654910

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 4 new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10007//console

This message is automatically generated.

> ScanResponse carries non-zero cellblock for CloseScanRequest (ScanRequest with close_scanner
= true)
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-11487
>                 URL: https://issues.apache.org/jira/browse/HBASE-11487
>             Project: HBase
>          Issue Type: Bug
>          Components: IPC/RPC, regionserver
>    Affects Versions: 0.96.2, 0.99.0, 2.0.0
>            Reporter: Shengzhe Yao
>            Assignee: Shengzhe Yao
>            Priority: Minor
>             Fix For: 2.0.0
>
>         Attachments: HBase_11487_v1.patch, HBase_11487_v2.patch
>
>
> After upgrading hbase from 0.94 to 0.96, we've found that our asynchbase client keep
throwing errors during normal scan. It turns out these errors are due to Scanner.close call
in asynchbase. Since asynchbase assumes the ScanResponse of CloseScannerRequest should never
carry any cellblocks, it will throw an exception if there is a violation.
> In the asynchbase client (1.5.0), it constructs a CloseScannerRequest in the following
way,  
>    ScanRequest.newBuilder()
>         .setScannerId(scanner_id)
>         .setCloseScanner(true)
>         .build();
> Note, it does not set numOfRows, which kind of make sense. Why a close scanner request
cares about number of rows to scan ?
> However, after narrowing down the CloseScannerRequest code path, it seems the issue is
on regionserver side. In RsRpcServices.scan, we always init numOfRows to scan to 1 and we
do this even for ScanRequest with close_scanner = true. This causes response for CloseScannerRequest
will carry a cellBlock (if scan stops before the end row and this could happen in many normal
scenarios)
> There are two fixes, either we always set numOfRows in asynchbase client side when constructing
a CloseScannerRequest or we fix the default value in the server side.
> From a hbase client side point of view, it seems make less sense that server will send
you a cellBlock for your close scanner request, unless the request explicitly asks for. 
> We've made the change in our server code and the asynchbase client errors goes away.

> In addition to this issue, I want to know if we have any specifications for our hbase
rpc. Like if close_scanner = true in ScanRequest and numOfRows is not set, ScanResponse guarantees
that there is no cellBlock in the response. Since we moved to protobuf and many fields are
optional for compatibility consideration, it might be helpful to have such specification which
helps people to develop code that depends on hbase rpc.     



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message