hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11433) LruBlockCache does not respect its configurable parameters
Date Mon, 30 Jun 2014 20:24:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048097#comment-14048097

Hadoop QA commented on HBASE-11433:

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  against trunk revision .
  ATTACHMENT ID: 12653191

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 3 warning messages.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 13 new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

     {color:red}-1 core tests{color}.  The patch failed these unit tests:

     {color:red}-1 core zombie tests{color}.  There are 2 zombie test(s): 	at org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompactionAfterWALSync(TestIOFencing.java:218)
	at org.apache.hadoop.hbase.regionserver.TestHRegion.testWritesWhileGetting(TestHRegion.java:3499)

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9899//console

This message is automatically generated.

> LruBlockCache does not respect its configurable parameters
> ----------------------------------------------------------
>                 Key: HBASE-11433
>                 URL: https://issues.apache.org/jira/browse/HBASE-11433
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.96.2, 0.99.0, 0.98.3
>            Reporter: Shengzhe Yao
>            Assignee: Shengzhe Yao
>             Fix For: 0.99.0, 0.96.3, 0.98.4
>         Attachments: HBase_11433_v1.patch, HBase_11433_v1.patch
> We've upgraded our production cluster from 0.94.15 to 0.96.2 few days ago and observed
increased GC frequency and occasionally full GC (we never had full GC before with G1 GC),
which leads to famous juliet pause...
> After digging into several HBase metrics, we've found that block cache used much higher
memory in 0.96. It turns out due to patch: HBASE-6312, which not only make a few block cache
parameter configurable, but also change their default values! It is obvious that we need to
set these parameters back to the old value before considering reduce block cache size or tuning
our GC. However, we are surprised that there is no change in regionserver side and we are
still observing high block cache usage.
> At the end of the day, it seems in CacheConfig.java, we initialize LruBlockCache with
default constructor: LruBlockCache(long maxSize, long blockSize), which underlying always
use the default values. We think this is a bug and we should always use another constructor:
LruBlockCache(long maxSize, long blockSize, boolean evictionThread, Configuration conf) in
> We made the change and tested on one of our servers, it works and now GC problem disappears.
Of course, we have to review our hbase and GC configurations and find the best configuration
under 0.96 for our application. But first, we feel the constructor misuse in CacheConfig.java
should be fixed.    

This message was sent by Atlassian JIRA

View raw message