hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-15561) See how G1GC works with MSLAB and chunk pool
Date Fri, 07 Oct 2016 11:29:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15544437#comment-15544437
] 

ramkrishna.s.vasudevan edited comment on HBASE-15561 at 10/7/16 11:29 AM:
--------------------------------------------------------------------------

Just wanted to update this JIRa with our findings and result over here. If things are discussed
we could close this JIRA.

There are two parts to this
-> testing trunk (default memstore) with *MSLAB and chunk pool ON* and without MSLAB with
G1GC configs
-> Next is to test G1GC with defaut memstore and offheap memstore (HBASE-15179). For now
this offheap is not in trunk. It is a POC work. Working on getting things committed to trunk.

cluster setup
==========
The set up is a single node set up with one RS and the client in the same machine. The machine
has 150G of RAM.

GC config:
========
export HBASE_OPTS="-XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=60
-XX:G1HeapWastePercent=20  -XX:G1MixedGCCountTarget=8"

<property>
            <name>hbase.regionserver.global.memstore.size</name>
  <value>0.42</value>
</property>


PE command:
===========
./hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --presplit=50 --size=150
--columns=50 --valueSize=200  --oneCon=true  --writeToWAL=false --bloomFilter=NONE --inmemoryCompaction=false
randomWrite 100

We think MSLAB and chunk pool is important mainly because when you move things to offheap
we need to ensure that we are managing the offheap buffers 
efficiently. It is very dangerous to allocate offheap buffers and keep them unclaimed which
will lead to offheap memory leak. Whereas in onheap buffers cases we
are able to manage it with the help of GC.
Hence it is always recommended for offheap memstore to go with MSLAB and chunk pool.

Perf readings
==========
For details on onheap and offheap memstore comparison with MSLAB and chunk pool pls refer
to these docs and various GC configs
https://docs.google.com/document/d/1fj5P8JeutQ-Uadb29ChDscMuMaJqaMNRI86C4k5S1rQ/edit
https://docs.google.com/document/d/1bYIkwNVVyYk8bgik-C5x_yc_2hWHyi2-9KMwEFWLfzY/edit

The above experiments were done on top of a POc branch where we had our own changes to support
offheap memstore including some PB changes.


With the current trunk we performed experiments with 50 threads and 100 threads and with 64
G heap memory.

With 100 threads and 10 cols per row pushed from a single client the setup with MSLAB and
chunk pool performs 13% better than the no MSLAB case.
With 50 threads and 10 cols per row pushed from a single client the setup with MSLAB and chunk
pool performs 11% better than the no MSLAB case.

PE completion time for 50 threads, 10 cols
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|1132.923 secs|1262 secs|

PE completion time for 100 threads, 10 cols
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|1947.330 secs|2214.201 secs|

No of GC pauses for 100 threads
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|403|475|

GC pause in secs for 100 threads
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|181.58 secs|239.9 secs|

Pls feel free to share your feedback or thoughts on this. It helps to see if there is something
to be tested or some configs needs to be changed and the tests has to be repeated. 


was (Author: ram_krish):
Just wanted to update this JIRa with our findings and result over here. If things are discussed
we could close this JIRA.

There are two parts to this
-> testing trunk (default memstore) with MSLAB and chunk pool ON and without MSLAB with
G1GC configs
-> Next is to test G1GC with defaut memstore and offheap memstore (HBASE-15179). For now
this offheap is not in trunk. It is a POC work. Working on getting things committed to trunk.

cluster setup
==========
The set up is a single node set up with one RS and the client in the same machine. The machine
has 150G of RAM.

GC config:
========
export HBASE_OPTS="-XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=60
-XX:G1HeapWastePercent=20  -XX:G1MixedGCCountTarget=8"

<property>
            <name>hbase.regionserver.global.memstore.size</name>
  <value>0.42</value>
</property>


PE command:
===========
./hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --presplit=50 --size=150
--columns=50 --valueSize=200  --oneCon=true  --writeToWAL=false --bloomFilter=NONE --inmemoryCompaction=false
randomWrite 100

We think MSLAB and chunk pool is important mainly because when you move things to offheap
we need to ensure that we are managing the offheap buffers 
efficiently. It is very dangerous to allocate offheap buffers and keep them unclaimed which
will lead to offheap memory leak. Whereas in onheap buffers cases we
are able to manage it with the help of GC.
Hence it is always recommended for offheap memstore to go with MSLAB and chunk pool.

Perf readings
==========
For details on onheap and offheap memstore comparison with MSLAB and chunk pool pls refer
to these docs and various GC configs
https://docs.google.com/document/d/1fj5P8JeutQ-Uadb29ChDscMuMaJqaMNRI86C4k5S1rQ/edit
https://docs.google.com/document/d/1bYIkwNVVyYk8bgik-C5x_yc_2hWHyi2-9KMwEFWLfzY/edit

The above experiments were done on top of a POc branch where we had our own changes to support
offheap memstore including some PB changes.


With the current trunk we performed experiments with 50 threads and 100 threads and with 64
G heap memory.

With 100 threads and 10 cols per row pushed from a single client the setup with MSLAB and
chunk pool performs 13% better than the no MSLAB case.
With 50 threads and 10 cols per row pushed from a single client the setup with MSLAB and chunk
pool performs 11% better than the no MSLAB case.

PE completion time for 50 threads, 10 cols
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|1132.923 secs|1262 secs|

PE completion time for 100 threads, 10 cols
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|1947.330 secs|2214.201 secs|

No of GC pauses for 100 threads
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|403|475|

GC pause in secs for 100 threads
||Default memstore with MSLAB ||Default memstore with NO MSLAB||
|181.58 secs|239.9 secs|

Pls feel free to share your feedback or thoughts on this. It helps to see if there is something
to be tested or some configs needs to be changed and the tests has to be repeated. 

> See how G1GC works with MSLAB and chunk pool
> --------------------------------------------
>
>                 Key: HBASE-15561
>                 URL: https://issues.apache.org/jira/browse/HBASE-15561
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>
> Based on the discussion in https://issues.apache.org/jira/browse/HBASE-14613, we need
to identify specifically how MSLAB and G1GC work. This sub-task is mainly to identify how
things work with G1GC and note down the observations. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message