hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13362) set max result size from client only (like caching)?
Date Mon, 30 Mar 2015 23:34:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387604#comment-14387604
] 

Josh Elser commented on HBASE-13362:
------------------------------------

{quote}
* remove any server only result sizing
* continue to use hbase.client.scanner.max.result.size but enforce it via the client only
(as the name implies anyway).
{quote}

Are you suggesting that the server shouldn't do anything with hbase.client.scanner.max.result.size
then? On its own, I agree that it's a bit goofy for that property to be used server-side (I
did a double-take the first time I saw it).

But, assuming I understand you correctly, if we remove that property from the server, we'd
have to rely on quotas to restrict memory used in a server for scans across all users, right?
Best as I can tell, there isn't a straightforward way to configure "RegionServer scans should
not exceed $X memory" as there is now (that parameter * number of rpc handlers or the scan
quota) which is a little worrisome at first glance as it would let clients use a lot of memory
out of the box if they set it in the Scan. I could just be ignorant of other configuration
properties though...

> set max result size from client only (like caching)?
> ----------------------------------------------------
>
>                 Key: HBASE-13362
>                 URL: https://issues.apache.org/jira/browse/HBASE-13362
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: Lars Hofhansl
>
> With the recent problems we've been seeing client/server result size mismatch, I was
thinking: Why was this not a problem with scanner caching?
> There are two reasons:
> # number of rows is easy to calculate (and we did it correctly)
> # caching is only controlled from the client, never set on the server alone
> We did fix both #1 and #2 in HBASE-13262.
> Still, I'd like to discuss the following:
> * default the client sent max result size to 2mb
> * remove any server only result sizing
> * continue to use hbase.client.scanner.max.result.size but enforce it via the client
only (as the name implies anyway).
> Comments? Concerns?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message