Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 10 Jun 2015 07:42:01 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12836657.1433883705000.35239.1433922121135@Atlassian.JIRA>
In-Reply-To: <JIRA.12836657.1433883705000@Atlassian.JIRA>
References: <JIRA.12836657.1433883705000@Atlassian.JIRA>
 <JIRA.12836657.1433883705073@arcas>
Subject: [jira] [Commented] (HBASE-13876) Improving performance of
 HeapMemoryManager
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580173#comment-14580173 ] 

Hadoop QA commented on HBASE-13876:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12738758/HBASE-13876-v2.patch
  against master branch at commit 4713fc6407450825b1907c833ca60d0cc604b5be.
  ATTACHMENT ID: 12738758

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 3 new or modified tests.

    {color:red}-1 javac{color}.  The patch appears to cause mvn compile goal to fail with Hadoop version 2.4.1.

    Compilation errors resume:
    [ERROR] COMPILATION ERROR : 
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:[1381,41] method create in class org.apache.hadoop.hbase.regionserver.HeapMemoryManager cannot be applied to given types;
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project hbase-server: Compilation failure
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:[1381,41] method create in class org.apache.hadoop.hbase.regionserver.HeapMemoryManager cannot be applied to given types;
[ERROR] required: org.apache.hadoop.conf.Configuration,org.apache.hadoop.hbase.regionserver.FlushRequester,org.apache.hadoop.hbase.Server,org.apache.hadoop.hbase.regionserver.RegionServerAccounting
[ERROR] found: org.apache.hadoop.conf.Configuration,org.apache.hadoop.hbase.regionserver.MemStoreFlusher,org.apache.hadoop.hbase.regionserver.HRegionServer
[ERROR] reason: actual and formal argument lists differ in length
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hbase-server
    

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/14362//console

This message is automatically generated.

> Improving performance of HeapMemoryManager
> ------------------------------------------
>
>                 Key: HBASE-13876
>                 URL: https://issues.apache.org/jira/browse/HBASE-13876
>             Project: HBase
>          Issue Type: Improvement
>          Components: hbase, regionserver
>    Affects Versions: 2.0.0, 1.0.1, 1.1.0, 1.1.1
>            Reporter: Abhilash
>            Assignee: Abhilash
>            Priority: Minor
>         Attachments: HBASE-13876-v1.patch, HBASE-13876-v2.patch, HBASE-13876.patch
>
>
> I am trying to improve the performance of DefaultHeapMemoryTuner by introducing some more checks. The current checks under which the DefaultHeapMemoryTuner works are very rare so I am trying to weaken these checks to improve its performance.
> Check current memstore size and current block cache size. If we are using less than 50% of currently available block cache size  we say block cache is sufficient and same for memstore. This check will be very effective when server is either load heavy or write heavy. Earlier version just waited for number of evictions / number of flushes to be zero which are very rare.
> Otherwise based on percent change in number of cache misses and number of flushes we increase / decrease memory provided for caching / memstore. After doing so, on next call of HeapMemoryTuner we verify that last change has indeed decreased number of evictions / flush ( combined). I am doing this analysis by comparing percent change (which is basically nothing but normalized derivative) of number of evictions and number of flushes during last two periods. The main motive for doing this was that if we have random reads then we will be having a lot of cache misses. But even after increasing block cache we wont be able to decrease number of cache misses and we will revert back and eventually we will not waste memory on block caches. This will also help us ignore random short term spikes in reads / writes.
>   


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)