hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15950) Fix memstore size estimates to be more tighter
Date Wed, 08 Jun 2016 03:17:21 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Enis Soztutar updated HBASE-15950:
----------------------------------
    Release Note: Made the object/array header sizes to be CompressedOops aware and fixed
object heap size calculations for important objects like KeyValue and ConcurrentSkipListMap
and . For heap sizes up to 32GB, depending on average cell sizes and total memstore size,
%10-50 reduction in memstore size and flushes and compactions might be observed. YMMV. Due
to more tighter than before size estimates, total heap space is expected to be utilized more,
slightly increasing the chance to get OOM if misconfigured. Set the environment property (not
hbase-site.xml) "hbase.memorylayout.use.unsafe" to false to disable. 

> Fix memstore size estimates to be more tighter
> ----------------------------------------------
>
>                 Key: HBASE-15950
>                 URL: https://issues.apache.org/jira/browse/HBASE-15950
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>             Fix For: 2.0.0
>
>         Attachments: Screen Shot 2016-06-02 at 8.48.27 PM.png, hbase-15950-v0.patch
>
>
> While testing something else, I was loading a region with a lot of data. Writing 30M
cells in 1M rows, with 1 byte values. 
> The memstore size turned out to be estimated as 4.5GB, while with the JFR profiling I
can see that we are using 2.8GB for all the objects in the memstore (KV + KV byte[] + CSLM.Node
+ CSLM.Index). 
> This obviously means that there is room in the write cache that we are not effectively
using. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message