hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions
Date Tue, 06 Dec 2016 08:08:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15724760#comment-15724760

Anoop Sam John commented on HBASE-16417:

Thanks for the great work..  You tried diff cases and captured all results nicely.
bq.This might be since now the block cache is used in full capacity, leaving the gc to struggle
with less free space. The chunk pool uses more space than the application can afford.
Your guess should be absolutely correct.  The inital heap occupancy percent for G1GC is configured?
 Default is 45%. In write only test of ours, we put 42% for global memstore size with this
factor in mind.. Now in ur tests 80% is what BC + memstore %.  In some other place also I
have commented, we must revist the default for BC and memstore size considering G1GC. The
working set size to be at least 80% is too much for G1GC. Against its spirit of predictable
GC pause
bq.and batching writes at the client side in buffer of size 10KB (vs no WAL and a buffer of
size 12MB in previous experiments)
Why the batching size at client is reduced?  10 KB?  That is too too small no?
bq.none‚Äč no compaction,
Here none case means what?
flattening index when #segments in pipeline is >2?

So it clearly says the 1st memstore and then HFiles optimization is helping..  So you will
raise it and provide ur internal patches there for initial ref?

> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>                 Key: HBASE-16417
>                 URL: https://issues.apache.org/jira/browse/HBASE-16417
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anastasia Braginsky
>            Assignee: Eshcar Hillel
>             Fix For: 2.0.0
>         Attachments: HBASE-16417-benchmarkresults-20161101.pdf, HBASE-16417-benchmarkresults-20161110.pdf,
HBASE-16417-benchmarkresults-20161123.pdf, HBASE-16417-benchmarkresults-20161205.pdf

This message was sent by Atlassian JIRA

View raw message