hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11887) Memory retention in branch-1; millions of instances of LiteralByteString for column qualifier and value
Date Wed, 03 Sep 2014 21:18:52 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120473#comment-14120473
] 

stack commented on HBASE-11887:
-------------------------------

jmap -histo:

{code}
 num     #instances         #bytes  class name
----------------------------------------------
   1:       3069883      780472352  [B
   2:       3501587      112050784  org.apache.hadoop.hbase.KeyValue
   3:       3441167       82588008  java.util.concurrent.ConcurrentSkipListMap$Node
   4:       1077906       68985984  org.apache.hadoop.hbase.protobuf.generated.ClientProtos$MutationProto$ColumnValue$QualifierValue
   5:       2372963       56951112  com.google.protobuf.LiteralByteString
   6:       1720444       41290656  java.util.concurrent.ConcurrentSkipListMap$Index
   7:          7024       18547960  [I
{code}

.... shows QualifierValue at 1/10th of that the numbers used be with no mention of the LiteralByteString
which used to have near equal pegging with [B instances.  Let me commit then since it makes
branch-1 YCSB-able (Previous it was not).  Will be back to try and make more improvements
but this is enough for one issue.

> Memory retention in branch-1; millions of instances of LiteralByteString for column qualifier
and value
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-11887
>                 URL: https://issues.apache.org/jira/browse/HBASE-11887
>             Project: HBase
>          Issue Type: Bug
>          Components: Protobufs
>    Affects Versions: 0.99.0
>            Reporter: stack
>            Assignee: stack
>            Priority: Critical
>             Fix For: 0.99.0, 2.0.0
>
>         Attachments: 11887.txt, Screen Shot 2014-09-03 at 10.18.58 AM.png
>
>
> Trying to test branch-1, I run out of mem pretty fast.  Looking at dumps, I see too many
instances of LiteralByteString.  Seem to be 'qualifiers' and 'values' out of pb QualifierValue...
and on up to the multi call into the server. Am having trouble finding how the retention is
being done... Filing issue in meantime while work on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message