hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eshcar Hillel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions
Date Thu, 27 Oct 2016 13:32:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611902#comment-15611902
] 

Eshcar Hillel commented on HBASE-16417:
---------------------------------------

The report of the first round of experiment is ready however I cannot attach it here.
Can anyone assign this subtask to me so I can attach files in it?
Meanwhile, I will attach it in the umbrella Jira.

The summary of the report is as follows --
Main difference in configuration vs previous benchmarks:
1. Since we run on a 48GB ram machine we allocate only 16GB to HBase (and not 32GB).
2. Saturation point was found when running 10 threads (and not 50); see more details in the
report.
3. We write 50GB (and not 150GB) just to have the experiments shorter since we run many different
settings.

First round of experiments compares different options (no-, index-, data-compaction) under
write-only workload with uniform keys distribution using PE. We see that up until the 95th
percentile all options are comparable. At the 99th percentile data compaction starts to lag
behind -- indeed in a uniform workload there is not much point in doing data compaction. The
overhead might stem from running SQM to determine which versions to retain. One way to close
this gap is to not run data compaction when there is no gain in it. A good policy should be
able to identify this with no extra cost.
At the 99.999th percentile index compaction also exhibits significant overhead. This might
be due to memory reclamation of temporary indices.

> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>
>                 Key: HBASE-16417
>                 URL: https://issues.apache.org/jira/browse/HBASE-16417
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anastasia Braginsky
>            Assignee: Anastasia Braginsky
>             Fix For: 2.0.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message