hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14978) Don't allow Multi to retain too many blocks
Date Thu, 17 Dec 2015 12:40:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061991#comment-15061991

Anoop Sam John commented on HBASE-14978:

Ok lets us target first some thing for 1.2 release.

bq.if (!(c instanceof BufferedDataBlockEncoder.SeekerState)) {
So you don't want to do this barrier check for DBE cases?  When DBEs we will be returning
ClonedSeekerState as Cells.  May be continue with what you were doing in old patch? (Consider
valueArray) So you can even do it for DBE case.   For PrefixTree also value buffer will be
the same byte[] which backs the HFileBlock I believe (Correct Ram?)
Here itself we are seeing the complexity and tomorrow if a new DBE comes in, we might break
the assumption here.. That was my worry.. The code is basically breaking the Cell notion..
I agree to your point that more accurate house keeping might be too much of overhead and we
need a fix for the OOME possibility.  Just saying my views. :-)

> Don't allow Multi to retain too many blocks
> -------------------------------------------
>                 Key: HBASE-14978
>                 URL: https://issues.apache.org/jira/browse/HBASE-14978
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 2.0.0, 1.2.0, 1.3.0
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>            Priority: Blocker
>             Fix For: 1.2.0, 3.0.0, 1.3.0
>         Attachments: HBASE-14978-branch-1.2.patch, HBASE-14978-v1.patch, HBASE-14978-v2.patch,
HBASE-14978-v3.patch, HBASE-14978-v4.patch, HBASE-14978.patch
> Scans and Multi's have limits on the total size of cells that can be returned. However
if those requests are not all pointing at the same blocks then the KeyValues can keep alive
a lot more data than their size.
> Take the following example:
> A multi with a list of 10000 gets to a fat row. Each column being returned in in a different
block. Each column is small 32 bytes or so.
> So the total cell size will be 32 * 10000 = ~320kb. However if each block is 128k then
total retained heap size will be almost 2gigs.

This message was sent by Atlassian JIRA

View raw message