hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions
Date Fri, 10 Mar 2017 10:11:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15904833#comment-15904833

Anoop Sam John commented on HBASE-16417:

Thanks for the nice perf tests and summary. 

bq. HBASE_OPTS="-XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=60
-XX:G1HeapWastePercent=20 -XX:G1MixedGCCountTarget=8
I have a concern here.  Using G1GC with InitiatingHeapOccupancyPercent(IHOP) of 60%. But in
mixed workload, the working size itself is 80% (40% for Memstore and 40% for BC). The GC impact
will be more here..Can we have a test with better GC tuning? Already we have load on GC and
the merge, as it needs more memory to work for the index copy (for merge), we might not see
better results?  Ya reading from more Segments, instead of 1, can be one reason for the 95th
percentile perf degrade.

bq.Since previous experiments show consistently weaker performance with on-heap mslabs we
focus on running experiments with no chunks pool​ and no mslabs
Above said GC config and HBase config can be an issue here also? The impact of this will be
more when u work with MSLAB as MSLAB chunks will be always fixed to process once created.
 With G1GC we should revist our defaults for BC and Memstore size.  We need to keep the sum
of both to be under the IHOP. Or else up IHOP. Then the advantage of G1GC itself is lost.
The default for IHOP is 45%

bq.We run in synchronous and asynch WAL modes.
How can the perf for CompactingMemstore be affected by WAL type? Any reason why u did test
for both types?  Any analysis here?

bq.Batching writes at the client side; buffer size is 10KB
Seems small buffer size. Any specific consideration in this selection? Just asking

Still not sure how the Eager mode reduced the #WAL files. 

> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>                 Key: HBASE-16417
>                 URL: https://issues.apache.org/jira/browse/HBASE-16417
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anastasia Braginsky
>            Assignee: Eshcar Hillel
>             Fix For: 2.0.0
>         Attachments: HBASE-16417-benchmarkresults-20161101.pdf, HBASE-16417-benchmarkresults-20161110.pdf,
HBASE-16417-benchmarkresults-20161123.pdf, HBASE-16417-benchmarkresults-20161205.pdf, HBASE-16417-benchmarkresults-20170309.pdf

This message was sent by Atlassian JIRA

View raw message