hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
Date Thu, 05 Feb 2015 05:12:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306648#comment-14306648
] 

Lars Hofhansl commented on HBASE-11544:
---------------------------------------

Awesome. That's exactly what I had in mind.

Long text and questions follows:

In the meanwhile we have been doing this: Set Scan.caching to a very high value, and set hbase.client.scanner.max.result.size
to 2mb.
(64k is way too small for 1ge or faster networks, takes only 0.5ms to send of 1ge, and is
the ballpark of latency which is at least 0.1-1ms typically, 2mb take 16ms and that's nice
to amortize the latency, i.e. for each 16ms we pay 0.1-1ms of latency, which is acceptable)

When we stream we only need to fill the bandwidth x latency product. With above number that
be only 12.5k (125mb/s bandwidth and 0.1ms roundtrip, if the latency was 1ms it would 125k,
and so on). That's for later.

So I do take back my statement about chunk sizes between 1-128k. Since we're *not* streaming
we need to amortize the latency incurred by each RPC request, and that means at large of chunk
size that we can get away with. I found that 2mb is good compromise.

I think hbase.client.scanner.max.result.size should be the default setting! That will already
go a long way. We should do this *now*, in all branches.

With that we always get batches of max 2mb shipped from the server to the client. In the extreme
this will only send a single row back per next() RPC if that row is > 2mb.

So we only have a problem when a single row is > 2mb. So that's already pretty good.

Now we need to deal with single large rows. Scanner caching is useless there, that's why we
have batching. That's a bit more tricky, since it's not clear to me what the client should
do. Should it always wait until it has a whole row as I said above? What if that is too large
for *it* handle. It seems we always have to at least allow for optional batching of partial
rows.
Single cells are even worse. I like your proposal because it handles even that, at least on
the server side. The client will still need to materialize the cell.

Now, given that we can solve the chunk size problem for many rows by setting hbase.client.scanner.max.result.size,
and that we probably need to allow the option to handle partial result at the client anyway,
is it worth engaging in this... Unless we do real streaming?


> [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even
if it means OOME
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-11544
>                 URL: https://issues.apache.org/jira/browse/HBASE-11544
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Priority: Critical
>              Labels: beginner
>
> Running some tests, I set hbase.client.scanner.caching=1000.  Dataset has large cells.
 I kept OOME'ing.
> Serverside, we should measure how much we've accumulated and return to the client whatever
we've gathered once we pass out a certain size threshold rather than keep accumulating till
we OOME.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message