hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eshcar Hillel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions
Date Mon, 05 Dec 2016 13:30:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15722264#comment-15722264

Eshcar Hillel commented on HBASE-16417:

Attached new benchmarks results.
This time we focus on running experiments with no chunk pool and no mslabs. We compare the
three policies discussed in HBASE-16851: *none* no compaction, *basic* flattening index, merge
only upon flush to disk (HBASE-17081), and *eager* data compaction.
You may review the results of cluster benchmarks again as they include experiments with the
new *basic* policy (Anastasia's code).

In write-only workload eager improves write amplification by 21%, while basic improves it
by 15%; Basic and eager outperform no compaction by 15% and 25%, resp. 
In read-write workload we show reduction in number of cache accesses and cache misses by basic
and eager w.r.t no compaction. We show modest improvement in average read latency of basic
and eager over none. With slower disks (HDD) reduction in cache misses will have more positive
affect on avg and read tail latencies.

We experimented with read operation optimization that speculatively scans memory-only components
first and only if the result is incomplete scans both memory and disk.  We ran the mixed workload
experiment for none, basic and eager with and without the optimization. 
The optimization improves avg read latency of none and basic by 10% and avg read latency of
eager by 7%. Reduction in cache accesses (70% less accesses), and cache misses (40%-45% less
misses) lead to this improvement. 

Finally we ran write-scan workload with size of scan is chosen uniformly from 1-5000. Scan
performance of all three policies are comparable with a modest improvement of basic and eager
over none (5%-7%).

> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>                 Key: HBASE-16417
>                 URL: https://issues.apache.org/jira/browse/HBASE-16417
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anastasia Braginsky
>            Assignee: Eshcar Hillel
>             Fix For: 2.0.0
>         Attachments: HBASE-16417-benchmarkresults-20161101.pdf, HBASE-16417-benchmarkresults-20161110.pdf,
HBASE-16417-benchmarkresults-20161123.pdf, HBASE-16417-benchmarkresults-20161205.pdf

This message was sent by Atlassian JIRA

View raw message