hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-14978) Don't allow Multi to retain too many blocks
Date Tue, 15 Dec 2015 08:33:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057602#comment-15057602
] 

Anoop Sam John edited comment on HBASE-14978 at 12/15/15 8:33 AM:
------------------------------------------------------------------

So with off heap Cells, withe very cell, we will treat it as coming from a new block?
In fact with off heap Cell, which are delivered from L2 off heap cache, we wont get any OOME
issue whatever be the #blocks one request touches. We avoid copying of block data from L2
off heap space to temp byte[].  So these Cells refer to the L2 cache space only.

bq.if (c.getRowArray() != lastBlock) {
Even with array backed cells also this wont work as expected when we use DBE.  In that case
every Cell will have new byte[] backing the key parts at least..  Now with Cells in place
every where, this kind of assumption taking is not correct IMHO.


was (Author: anoop.hbase):
So with off heap Cells, withe very cell, we will treat it as coming from a new block?

bq.if (c.getRowArray() != lastBlock) {
Even with array backed cells also this wont work as expected when we use DBE.  In that case
every Cell will have new byte[] backing the key parts at least..  Now with Cells in place
every where, this kind of assumption taking is not correct IMHO.

> Don't allow Multi to retain too many blocks
> -------------------------------------------
>
>                 Key: HBASE-14978
>                 URL: https://issues.apache.org/jira/browse/HBASE-14978
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 2.0.0, 1.2.0, 1.3.0
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>            Priority: Critical
>         Attachments: HBASE-14978-v1.patch, HBASE-14978.patch
>
>
> Scans and Multi's have limits on the total size of cells that can be returned. However
if those requests are not all pointing at the same blocks then the KeyValues can keep alive
a lot more data than their size.
> Take the following example:
> A multi with a list of 10000 gets to a fat row. Each column being returned in in a different
block. Each column is small 32 bytes or so.
> So the total cell size will be 32 * 10000 = ~320kb. However if each block is 128k then
total retained heap size will be almost 2gigs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message