hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13527) The default value for hbase.client.scanner.max.result.size is never actually set on Scans
Date Wed, 22 Apr 2015 04:11:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14506384#comment-14506384
] 

Hadoop QA commented on HBASE-13527:
-----------------------------------

{color:green}+1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12727088/HBASE-13527-v1.patch
  against master branch at commit 07b8a7b264bc6d91fa63243b2e9cd5527176a6f5.
  ATTACHMENT ID: 12727088

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 4 new or modified
tests.

    {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions
(2.4.1 2.5.2 2.6.0)

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 protoc{color}.  The applied patch does not increase the total number of
protoc compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 checkstyle{color}.  The applied patch does not increase the total number
of checkstyle errors

    {color:green}+1 findbugs{color}.  The patch does not introduce any  new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/13766//testReport/
Release Findbugs (version 2.0.3) 	warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13766//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/13766//artifact/patchprocess/checkstyle-aggregate.html

  Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/13766//console

This message is automatically generated.

> The default value for hbase.client.scanner.max.result.size is never actually set on Scans
> -----------------------------------------------------------------------------------------
>
>                 Key: HBASE-13527
>                 URL: https://issues.apache.org/jira/browse/HBASE-13527
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.12, 1.2.0
>            Reporter: Jonathan Lawlor
>         Attachments: HBASE-13527-v1.patch
>
>
> Now that max result size is driven from the client side like caching (HBASE-13362), we
also need to set Scan.maxResultSize to the default value of hbase.client.scanner.max.result.size
which is never performed. I think this has gone unnoticed because the server used to read
the configuration hbase.client.scanner.max.result.size for itself, but now we expect the serialized
Scan sent from the client side to contain this information. Realistically this should have
been set on the Scans even before HBASE-13362, it's surprising that it's not as the scanner
code seems to indicate otherwise.
> Ultimately, the end result is that, by default, scan RPC's are limited by hbase.server.scanner.max.result.size
(note this is the new server side config not the client side config) which has a default value
of 100 MB. The scan RPC's should instead be limited by hbase.client.scanner.max.result.size
which has a default value of 2 MB.
> The reason why this issue occurs is because, by default, a new Scan() initializes Scan.maxResultSize
to -1. This initial value of -1 will never be changed unless Scan#setMaxResultSize() is called.
In the event that this value is not changed, the Scan that is serialized and sent to the server
will also have Scan.maxResultSize = -1. Then, when the server is deciding what size limit
should be enforced, it sees that Scan.maxResultSize = -1 so it uses the most relaxed size
restriction possible, which is hbase.server.scanner.max.result.size (default value 100 MB).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message