hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Latham (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-1959) Compress tables during 0.19 to 0.20 migration
Date Fri, 06 Nov 2009 23:32:32 GMT
Compress tables during 0.19 to 0.20 migration

                 Key: HBASE-1959
                 URL: https://issues.apache.org/jira/browse/HBASE-1959
             Project: Hadoop HBase
          Issue Type: New Feature
            Reporter: Dave Latham
         Attachments: 1959-branch0.20.patch

Current migration has no support for compression.  You can always enable compression after
migration and it should take effect after the next major compaction.  For anyone else who
may still have a significant amount of compressed data in 0.19, this might prove useful.

We have a large amount of data stored in 0.19 under GZip compression, at an effective 10x
rate.  Out of concern for migration speed and the risk of using more capacity than our cluster
has, we didn't want to first write out all the data uncompressed before enabling LZO compression.

I'm attaching a small patch that allowed us to migrate the data directly to a compressed table
format.  To use it, add a migrate.compression property to hbase-site.xml with a value of LZO
or GZ, and it should compress all tables during migration.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message