hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14978) Don't allow Multi to retain too many blocks
Date Wed, 29 Mar 2017 03:46:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946482#comment-15946482

Anoop Sam John commented on HBASE-14978:

Reading this part of code again
if (c instanceof ByteBufferCell) {
          ByteBufferCell bbCell = (ByteBufferCell) c;
          ByteBuffer bb = bbCell.getValueByteBuffer();
          if (bb != lastBlock) {
            lastBlock = bb;
This is specific case when the cell is read from BucketCache. The cell will be backed by DBB
in the Bucket Cache.  Each of these BB in bucket cache is of capacity 4 MB!  It is possible
to have multiple buckets and blocks within a BB.  The code above will take the response block
size as 4 MB!   This will make the response to be send back immediately.  This might not be
correct IMO.  If this was counting 64 KB block size ya, that may be ok but 4 MB is too much.
 What do you say [~eclark]?
cc [~saint.ack@gmail.com], [~ram_krish]

> Don't allow Multi to retain too many blocks
> -------------------------------------------
>                 Key: HBASE-14978
>                 URL: https://issues.apache.org/jira/browse/HBASE-14978
>             Project: HBase
>          Issue Type: Improvement
>          Components: io, IPC/RPC, regionserver
>    Affects Versions: 2.0.0, 1.2.0, 1.3.0
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>            Priority: Blocker
>             Fix For: 2.0.0, 1.2.0, 1.3.0
>         Attachments: HBASE-14978-branch-1.2.patch, HBASE-14978-branch-1.2.patch, HBASE-14978-branch-1.2.patch,
HBASE-14978-branch-1.2.patch, HBASE-14978.patch, HBASE-14978-v1.patch, HBASE-14978-v2.patch,
HBASE-14978-v3.patch, HBASE-14978-v4.patch
> Scans and Multi's have limits on the total size of cells that can be returned. However
if those requests are not all pointing at the same blocks then the KeyValues can keep alive
a lot more data than their size.
> Take the following example:
> A multi with a list of 10000 gets to a fat row. Each column being returned in in a different
block. Each column is small 32 bytes or so.
> So the total cell size will be 32 * 10000 = ~320kb. However if each block is 128k then
total retained heap size will be almost 2gigs.

This message was sent by Atlassian JIRA

View raw message