Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 27 Oct 2016 13:32:58 +0000 (UTC)
From: "Eshcar Hillel (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12997265.1471256753000.101320.1477575178457@Atlassian.JIRA>
In-Reply-To: <JIRA.12997265.1471256753000@Atlassian.JIRA>
References: <JIRA.12997265.1471256753000@Atlassian.JIRA> <JIRA.12997265.1471256753724@arcas>
Subject: [jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for
 Flattening and Compactions
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 27 Oct 2016 13:33:00 -0000


    [ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611902#comment-15611902 ] 

Eshcar Hillel commented on HBASE-16417:
---------------------------------------

The report of the first round of experiment is ready however I cannot attach it here.
Can anyone assign this subtask to me so I can attach files in it?
Meanwhile, I will attach it in the umbrella Jira.

The summary of the report is as follows --
Main difference in configuration vs previous benchmarks:
1. Since we run on a 48GB ram machine we allocate only 16GB to HBase (and not 32GB).
2. Saturation point was found when running 10 threads (and not 50); see more details in the report.
3. We write 50GB (and not 150GB) just to have the experiments shorter since we run many different settings.

First round of experiments compares different options (no-, index-, data-compaction) under write-only workload with uniform keys distribution using PE. We see that up until the 95th percentile all options are comparable. At the 99th percentile data compaction starts to lag behind -- indeed in a uniform workload there is not much point in doing data compaction. The overhead might stem from running SQM to determine which versions to retain. One way to close this gap is to not run data compaction when there is no gain in it. A good policy should be able to identify this with no extra cost.
At the 99.999th percentile index compaction also exhibits significant overhead. This might be due to memory reclamation of temporary indices.

> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>
>                 Key: HBASE-16417
>                 URL: https://issues.apache.org/jira/browse/HBASE-16417
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anastasia Braginsky
>            Assignee: Anastasia Braginsky
>             Fix For: 2.0.0
>
>


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)