hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
Date Wed, 22 Feb 2017 16:24:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878626#comment-15878626

Anoop Sam John commented on HBASE-17647:

Thanks for the review Stack.. Now we have ByteBufferKeyValue which can be backed by both on
heap as well as off heap BB.  So the change in the heapSize calc.
Ya I will rename the params and method as needed..  Changed some places and missed some may
Ya old 'size' is what dataSize now.  dataSize might be in on heap or off heap..  heapSize
is tracks total heap space occupied by all cells.  This includes cell POJO heap overheads
and if a cell is on heap, the total length of this cells key + value which resides in on heap
area any way. 

> OffheapKeyValue#heapSize() implementation is wrong
> --------------------------------------------------
>                 Key: HBASE-17647
>                 URL: https://issues.apache.org/jira/browse/HBASE-17647
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0
>         Attachments: HBASE-17647.patch, HBASE-17647_V2.patch
> We consider the key and data lengths also even though the data is actually in off heap
area.  We should correct it.
> The impact will be at ScannerContext limit tracking where we use heapSize of cells to
account the result size.  So my proposal is to consider the cells length and heap size in
Limit tracking and accounting.  We have a maxResultSize which defaults to 2MB.  When the sum
of all cell's data size reaches 'maxResultSize'  OR the sum of all cell's heap size reaches
'maxResultSize' , we need to send back the RPC response

This message was sent by Atlassian JIRA

View raw message