hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5778) Turn on WAL compression by default
Date Fri, 13 Apr 2012 04:57:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253110#comment-13253110
] 

Hadoop QA commented on HBASE-5778:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12522527/5778-addendum.txt
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 6 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.replication.TestReplication
                  org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
                  org.apache.hadoop.hbase.replication.TestMasterReplication

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1507//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1507//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1507//console

This message is automatically generated.
                
> Turn on WAL compression by default
> ----------------------------------
>
>                 Key: HBASE-5778
>                 URL: https://issues.apache.org/jira/browse/HBASE-5778
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Jean-Daniel Cryans
>            Assignee: Lars Hofhansl
>            Priority: Blocker
>             Fix For: 0.94.0, 0.96.0
>
>         Attachments: 5778-addendum.txt, 5778.addendum, HBASE-5778.patch
>
>
> I ran some tests to verify if WAL compression should be turned on by default.
> For a use case where it's not very useful (values two order of magnitude bigger than
the keys), the insert time wasn't different and the CPU usage 15% higher (150% CPU usage VS
130% when not compressing the WAL).
> When values are smaller than the keys, I saw a 38% improvement for the insert run time
and CPU usage was 33% higher (600% CPU usage VS 450%). I'm not sure WAL compression accounts
for all the additional CPU usage, it might just be that we're able to insert faster and we
spend more time in the MemStore per second (because our MemStores are bad when they contain
tens of thousands of values).
> Those are two extremes, but it shows that for the price of some CPU we can save a lot.
My machines have 2 quads with HT, so I still had a lot of idle CPUs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message