hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yanping Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14613) Remove MemStoreChunkPool?
Date Fri, 25 Mar 2016 05:47:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211455#comment-15211455

Yanping Wang commented on HBASE-14613:

Jingcheng, pointed this JIRA to me. I am kind of curious, have you guys tried to tune G1GC
for that "significant amount of memory basically pinned in old gen forever" case?
For example: by default in JDK8, the -XX:InitiatingHeapOccupancyPercent is 45% that means
G1 is trying to keep the "balance" around 45% heap full when starts concurrent marking cycle
for Mixed GC.
if the "significant amount of memory pinned in old gen forever" caused heap Occupancy Percent
more than 45% of total heap, then G1 will sure start concurrent marking cycle one after another,
which I know it is mostly not that necessary and hurt throughput. in that case, I tried to
increase -XX:InitiatingHeapOccupancyPercent=65 depends on how much (significant amount of
memory pinned in old gen forever) vs. total heap ratio. we can calculate that actually based
on test cases.

also notice since we increased the "balance" point up, we have to increase the speed of concurrent
marking cycle so it can finish faster to let Mixed GC to start clean up the rest of heap without
causing full GC. to do that, I also increase -XX:ConcGCThreads to be 2/3 of parallelGC threads
(default is 1/4).

I ran HBase with 100GB and 120GB heap before with G1GC, it worked fine. 
The good news is: jdk9 will have automatic InitiatingHeapOccupancyPercent adjustment based
on its runtime heap profiling data, so it can detect the live forever objects size and InitiatingHeapOccupancyPercent
on fly.

> Remove MemStoreChunkPool?
> -------------------------
>                 Key: HBASE-14613
>                 URL: https://issues.apache.org/jira/browse/HBASE-14613
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: Lars Hofhansl
>            Priority: Minor
>         Attachments: 14613-0.98.txt, gc.png, writes.png
> I just stumbled across MemStoreChunkPool. The idea behind is to reuse chunks of allocations
rather than letting the GC handle this.
> Now, it's off by default, and it seems to me to be of dubious value. I'd recommend just
removing it.

This message was sent by Atlassian JIRA

View raw message