hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-5778) Turn on WAL compression by default
Date Thu, 12 Apr 2012 22:19:18 GMT
Turn on WAL compression by default

                 Key: HBASE-5778
                 URL: https://issues.apache.org/jira/browse/HBASE-5778
             Project: HBase
          Issue Type: Improvement
            Reporter: Jean-Daniel Cryans
            Priority: Blocker
             Fix For: 0.94.0, 0.96.0

I ran some tests to verify if WAL compression should be turned on by default.

For a use case where it's not very useful (values two order of magnitude bigger than the keys),
the insert time wasn't different and the CPU usage 15% higher (150% CPU usage VS 130% when
not compressing the WAL).

When values are smaller than the keys, I saw a 38% improvement for the insert run time and
CPU usage was 33% higher (600% CPU usage VS 450%). I'm not sure WAL compression accounts for
all the additional CPU usage, it might just be that we're able to insert faster and we spend
more time in the MemStore per second (because our MemStores are bad when they contain tens
of thousands of values).

Those are two extremes, but it shows that for the price of some CPU we can save a lot. My
machines have 2 quads with HT, so I still had a lot of idle CPUs.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message