hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14978) Don't allow Multi to retain too many blocks
Date Tue, 15 Dec 2015 10:16:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057783#comment-15057783

ramkrishna.s.vasudevan commented on HBASE-14978:

bq.We won't get an OOME instead we'll kill the cache and be un-able to read anything from
Just wanted to understand your concern here.  In HBase branch-1.3 even if we configure the
BucketCache (L2) offheap every time we bring in the block onheap before serving any read request.
So in case of multi Gets there could be n number of blocks being copied to the onheap and
all those will have references in the Result till the results are transferred to the client.
So the heap can grow upto 2GB as per  your decription. 
But in the latest trunk this copy to onheap will never happen so the Cells that are created
in the Results will only refer directly to the offheap area and as you said we do the ref
counting of these blocks in offheap.
The only thing that could happen now is that till all the Results could be transferred to
the client we may hold the block and so any such blocks cannot be evicted till the reference
is cleared. 

> Don't allow Multi to retain too many blocks
> -------------------------------------------
>                 Key: HBASE-14978
>                 URL: https://issues.apache.org/jira/browse/HBASE-14978
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 2.0.0, 1.2.0, 1.3.0
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>            Priority: Critical
>         Attachments: HBASE-14978-v1.patch, HBASE-14978-v2.patch, HBASE-14978.patch
> Scans and Multi's have limits on the total size of cells that can be returned. However
if those requests are not all pointing at the same blocks then the KeyValues can keep alive
a lot more data than their size.
> Take the following example:
> A multi with a list of 10000 gets to a fat row. Each column being returned in in a different
block. Each column is small 32 bytes or so.
> So the total cell size will be 32 * 10000 = ~320kb. However if each block is 128k then
total retained heap size will be almost 2gigs.

This message was sent by Atlassian JIRA

View raw message