Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 16 Jun 2015 18:44:01 +0000 (UTC)
From: "Elliott Clark (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12836657.1433883705000.86780.1434480241610@Atlassian.JIRA>
In-Reply-To: <JIRA.12836657.1433883705000@Atlassian.JIRA>
References: <JIRA.12836657.1433883705000@Atlassian.JIRA>
 <JIRA.12836657.1433883705073@arcas>
Subject: [jira] [Commented] (HBASE-13876) Improving performance of
 HeapMemoryManager
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HBASE-13876?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D145=
88552#comment-14588552 ]=20

Elliott Clark commented on HBASE-13876:
---------------------------------------

+1. lgtm

> Improving performance of HeapMemoryManager
> ------------------------------------------
>
>                 Key: HBASE-13876
>                 URL: https://issues.apache.org/jira/browse/HBASE-13876
>             Project: HBase
>          Issue Type: Improvement
>          Components: hbase, regionserver
>    Affects Versions: 2.0.0, 1.0.1, 1.1.0, 1.1.1
>            Reporter: Abhilash
>            Assignee: Abhilash
>            Priority: Minor
>         Attachments: HBASE-13876-v2.patch, HBASE-13876-v3.patch, HBASE-13=
876-v4.patch, HBASE-13876-v5.patch, HBASE-13876-v6.patch, HBASE-13876-v7.pa=
tch, HBASE-13876.patch
>
>
> I am trying to improve the performance of DefaultHeapMemoryTuner by intro=
ducing some more checks. The current checks under which the DefaultHeapMemo=
ryTuner works are very rare so I am trying to weaken these checks to improv=
e its performance.
> Check current memstore size and current block cache size. For say if we a=
re using less than 50% of currently available block cache size  we say bloc=
k cache is sufficient and same for memstore. This check will be very effect=
ive when server is either load heavy or write heavy. Earlier version just w=
aited for number of evictions / number of flushes to be zero which are very=
 rare.
> Otherwise based on percent change in number of cache misses and number of=
 flushes we increase / decrease memory provided for caching / memstore. Aft=
er doing so, on next call of HeapMemoryTuner we verify that last change has=
 indeed decreased number of evictions / flush either of which it was expect=
ed to do. We also check that it does not make the other (evictions / flush)=
 increase much. I am doing this analysis by comparing percent change (which=
 is basically nothing but normalized derivative) of number of evictions and=
 number of flushes during last two periods. The main motive for doing this =
was that if we have random reads then we will be having a lot of cache miss=
es. But even after increasing block cache we wont be able to decrease numbe=
r of cache misses and we will revert back and eventually we will not waste =
memory on block caches. This will also help us ignore random short term spi=
kes in reads / writes. I have also tried to take care not to tune memory if=
 do do not have enough hints as unnecessary tuning my slow down the system.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)