hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jingcheng Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14613) Remove MemStoreChunkPool?
Date Wed, 23 Mar 2016 06:33:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207955#comment-15207955
] 

Jingcheng Du commented on HBASE-14613:
--------------------------------------

I did some similar tests (YCSB, 1TB dataset, 1 HMaster and 3 Region servers) and post the
result in HBASE-15180. We can see GC got improved a lot when chunk pool is enabled.


| |  mslab-pool-on | mslab-on | mslab-off |
| throughput|  242801  | 240464  | 225679  |
| latency(us)|  486  | 483  | 704  |
| gc pause(ms)|  8440 | 12837  | 17131  |

I used 2MB for the chunk size during the test.

Speaking of G1GC, G1GC splits the heap into 2000 regions. In my test (the heap size is 64GB),
each region will be 32MB. 
If the chunk pool is off, there might be fragments after minor gc (2MB is a fragment comparing
to 32MB, and the fragments are compacted in mixed gc and full gc).
If the chunk size is increased to 16MB or larger, I believe the performance can be improved
comparing to 2MB chunk size

> Remove MemStoreChunkPool?
> -------------------------
>
>                 Key: HBASE-14613
>                 URL: https://issues.apache.org/jira/browse/HBASE-14613
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: Lars Hofhansl
>            Priority: Minor
>         Attachments: 14613-0.98.txt, gc.png, writes.png
>
>
> I just stumbled across MemStoreChunkPool. The idea behind is to reuse chunks of allocations
rather than letting the GC handle this.
> Now, it's off by default, and it seems to me to be of dubious value. I'd recommend just
removing it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message