Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 22 Apr 2015 03:09:59 +0000 (UTC)
From: "Jonathan Lawlor (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12822865.1429662399000.70450.1429672199230@Atlassian.JIRA>
In-Reply-To: <JIRA.12822865.1429662399000@Atlassian.JIRA>
References: <JIRA.12822865.1429662399000@Atlassian.JIRA>
 <JIRA.12822865.1429662399117@arcas>
Subject: [jira] [Updated] (HBASE-13527) The default value for
 hbase.client.scanner.max.result.size is never actually set on Scans
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HBASE-13527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Lawlor updated HBASE-13527:
------------------------------------
    Description: 
Now that max result size is driven from the client side like caching (HBASE-13362), we also need to set Scan.maxResultSize to the default value of hbase.client.scanner.max.result.size which is never performed. I think this has gone unnoticed because the server used to read the configuration hbase.client.scanner.max.result.size for itself, but now we expect the serialized Scan sent from the client side to contain this information. Realistically this should have been set on the Scans even before HBASE-13362, it's surprising that it's not as the scanner code seems to indicate otherwise.

Ultimately, the end result is that, by default, scan RPC's are limited by hbase.server.scanner.max.result.size (note this is the new server side config not the client side config) which has a default value of 100 MB. The scan RPC's should instead be limited by hbase.client.scanner.max.result.size which has a default value of 2 MB.

The reason why this issue occurs is because, by default, a new Scan() initializes Scan.maxResultSize to -1. This initial value of -1 will never be changed unless Scan#setMaxResultSize() is called. In the event that this value is not changed, the Scan that is serialized and sent to the server will also have Scan.maxResultSize = -1. Then, when the server is deciding what size limit should be enforced, it sees that Scan.maxResultSize = -1 so it uses the most relaxed size restriction possible, which is hbase.server.scanner.max.result.size (default value 100 MB).

  was:
Now that max result size is driven from the client side like caching (HBASE-13362), we also need to set Scan.maxResultSize to the default value of hbase.client.scanner.max.result.size which is never performed. I think this has gone unnoticed because the server used to read the configuration hbase.client.scanner.max.result.size for itself, but now we expect the serialized Scan sent from the client side to contain this information. Realistically this should have been set on the Scans even before HBASE-13362, it's surprising that it's not as the scanner code seems to indicate otherwise.

Ultimately, the end result is that, by default, scan RPC's are limited by hbase.server.scanner.max.result.size (note this is the new server side config not the client side config) which has a default value of 100 MB. The scan RPC's should instead be limited by hbase.client.scanner.max.result.size which has a default value of 2 MB.


> The default value for hbase.client.scanner.max.result.size is never actually set on Scans
> -----------------------------------------------------------------------------------------
>
>                 Key: HBASE-13527
>                 URL: https://issues.apache.org/jira/browse/HBASE-13527
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.12, 1.2.0
>            Reporter: Jonathan Lawlor
>         Attachments: HBASE-13527-v1.patch
>
>
> Now that max result size is driven from the client side like caching (HBASE-13362), we also need to set Scan.maxResultSize to the default value of hbase.client.scanner.max.result.size which is never performed. I think this has gone unnoticed because the server used to read the configuration hbase.client.scanner.max.result.size for itself, but now we expect the serialized Scan sent from the client side to contain this information. Realistically this should have been set on the Scans even before HBASE-13362, it's surprising that it's not as the scanner code seems to indicate otherwise.
> Ultimately, the end result is that, by default, scan RPC's are limited by hbase.server.scanner.max.result.size (note this is the new server side config not the client side config) which has a default value of 100 MB. The scan RPC's should instead be limited by hbase.client.scanner.max.result.size which has a default value of 2 MB.
> The reason why this issue occurs is because, by default, a new Scan() initializes Scan.maxResultSize to -1. This initial value of -1 will never be changed unless Scan#setMaxResultSize() is called. In the event that this value is not changed, the Scan that is serialized and sent to the server will also have Scan.maxResultSize = -1. Then, when the server is deciding what size limit should be enforced, it sees that Scan.maxResultSize = -1 so it uses the most relaxed size restriction possible, which is hbase.server.scanner.max.result.size (default value 100 MB).


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)