hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12976) Default hbase.client.scanner.max.result.size
Date Thu, 05 Feb 2015 17:54:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307647#comment-14307647

Lars Hofhansl commented on HBASE-12976:

Actually, with this in place, we can up the default scanner caching to Long.MAX_VALUE. It'll
be capped at 2mb.

The only reason to set scanner caching now is when the client knows how many rows it is going
to need or expect.
So scanner caching is now only used to *limit* the result size based on semantic knowledge
of the client, but no longer to optimize the batch sizes.
Might even rename caching to rowLimit or something.

There might be effects with coprocessors that I would need to double check, so I won't do
that part in this jira.

> Default hbase.client.scanner.max.result.size
> --------------------------------------------
>                 Key: HBASE-12976
>                 URL: https://issues.apache.org/jira/browse/HBASE-12976
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>             Fix For: 2.0.0, 1.0.1, 0.94.27, 0.98.11
>         Attachments: 12976.txt
> Setting scanner caching is somewhat of a black art. It's hard to estimate ahead of time
how large the result set will be.
> I propose we hbase.client.scanner.max.result.size to 2mb. That is good compromise between
performance and buffer usage on typical networks (avoiding OOMs when the caching was chosen
too high).
> To an HTable client this is completely transparent.

This message was sent by Atlassian JIRA

View raw message