hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang Xie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8163) MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
Date Thu, 14 Nov 2013 10:37:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822308#comment-13822308

Liang Xie commented on HBASE-8163:

I did a apple-to-apple test with this patch. Here is the test result just for refer:

Env: Xmx=Xms=8G,Xmn1G, memestore  lower/upper limit = 0.2/0.3
each test began with empty table, then write 20million records each with three fields, and
each field has 200 bytes:
1)original config(-XX:PretenureSizeThreshold=4m)
YGC     YGCT    FGC    FGCT     GCT
 6970  318.592     8    0.884  319.476
2)set hbase.hregion.memstore.mslab.chunksize to 4194320(-XX:PretenureSizeThreshold=4m)
YGC     YGCT    FGC    FGCT     GCT
 6973  253.891     8    0.522  254.413
3)set -XX:PretenureSizeThreshold=2097088(hbase.hregion.memstore.mslab.chunksize is by default
6960  260.642     8    1.427  262.069
4)set hbase.hregion.memstore.chunkpool.maxsize=0.6, means enable MemStoreChunkPool feature(log
said maxCount=706), -XX:PretenureSizeThreshold=2097088(hbase.hregion.memstore.mslab.chunksize
is by default 2M)
7028  258.598     2    0.401  258.999

To me, this MemStorechunkPool feature is useful for heavy FGC scenarios caused by write request.
 If the YGC is the big hurt by write request, personally i recommend to config "hbase.hregion.memstore.mslab.chunksize"
or "-XX:PretenureSizeThreshold", considering risk:)

Hope this test result will be helpful for some guys

> MemStoreChunkPool: An improvement for JAVA GC when using MSLAB
> --------------------------------------------------------------
>                 Key: HBASE-8163
>                 URL: https://issues.apache.org/jira/browse/HBASE-8163
>             Project: HBase
>          Issue Type: New Feature
>          Components: Performance, regionserver
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>             Fix For: 0.98.0, 0.95.0
>         Attachments: hbase-0.95-8163v6.patch, hbase-8163v1.patch, hbase-8163v2.patch,
hbase-8163v3.patch, hbase-8163v4.patch, hbase-8163v5.patch, hbase-8163v6.patch
> *Usage:*
> Disable pool(default): configure "hbase.hregion.memstore.chunkpool.maxsize" as 0
> Enable pool: configure "hbase.hregion.memstore.chunkpool.maxsize" as a percentage of
global memstore size (between 0.0 and 1.0, recommend to set it as the gap between min and
max sizes of memstore, e.g. 0.5)
> *Background*:
> When we use mslab,we will copy the keyvalue together in a structure called *MemStoreLAB$Chunk*,
therefore we could decrease the heap fragment. 
> *Problem*:
> When one chunk is full, we will create a new chunk, and then the old chunk will be reclaimed
by JVM if no reference to it.
> Mostly the chunk object would be promoted when doing Young GC, cause increasing the cost
of YGC 
> When a Chunk object has no reference? It should meet two following condition:
> 1.Memstore which this chunk belongs to is flushed
> 2.No scanner is opening on the memstore which this chunk belongs to
> *Solution:*
> 1.Create a chunk pool to manage the no-reference chunks, instead of being reclaimed by
> 2.When a Chunk has no reference, put it back to the pool
> 3.The pool has a max capacity, it will skip the chunks when achieve the max size
> 4.When we need new Chunk to store KeyValue, get it from the pool if exists, else create
new one by pool, so we could be able to reuse the old chunks
> *Test results:*
> Environment:
> hbase-version:0.94
> -Xms4G -Xmx4G -Xmn2G
> Row size=50 bytes, Value size=1024 bytes
> 50 concurrent theads per client, insert 10,000,000 rows
> Before:
> Avg write request per second:12953
> After testing, final result of jstat -gcutil :
> 747 36.503 48 2.492 38.995
> After:
> Avg write request per second:14025
> After testing, final result of jstat -gcutil :
> 711 20.344 4 0.284 20.628
> *Improvement: YGC 40+%; WPS 5%+*
> review board :
> https://reviews.apache.org/r/10056/

This message was sent by Atlassian JIRA

View raw message