hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17647) OffheapKeyValue#heapSize() implementation is wrong
Date Thu, 23 Feb 2017 11:36:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880290#comment-15880290
] 

Hudson commented on HBASE-17647:
--------------------------------

FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2556 (See [https://builds.apache.org/job/HBase-Trunk_matrix/2556/])
HBASE-17647 OffheapKeyValue#heapSize() implementation is wrong. (anoopsamjohn: rev ff045cab84ada47ae2d10015649cca8f6edb76ac)
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/CellUtil.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.java
* (edit) hbase-prefix-tree/src/main/java/org/apache/hadoop/hbase/codec/prefixtree/PrefixTreeSeeker.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/NoLimitScannerContext.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScannerContext.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/ByteBufferKeyValue.java


> OffheapKeyValue#heapSize() implementation is wrong
> --------------------------------------------------
>
>                 Key: HBASE-17647
>                 URL: https://issues.apache.org/jira/browse/HBASE-17647
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0
>
>         Attachments: HBASE-17647.patch, HBASE-17647_V2.patch, HBASE-17647_V3.patch
>
>
> We consider the key and data lengths also even though the data is actually in off heap
area.  We should correct it.
> The impact will be at ScannerContext limit tracking where we use heapSize of cells to
account the result size.  So my proposal is to consider the cells length and heap size in
Limit tracking and accounting.  We have a maxResultSize which defaults to 2MB.  When the sum
of all cell's data size reaches 'maxResultSize'  OR the sum of all cell's heap size reaches
'maxResultSize' , we need to send back the RPC response



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message