hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Latham (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1959) Compress tables during 0.19 to 0.20 migration
Date Fri, 06 Nov 2009 23:32:32 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Dave Latham updated HBASE-1959:

    Attachment: 1959-branch0.20.patch

This patch modifies Migrate.java to check a config parameter and write tables in a compressed
format directly.

> Compress tables during 0.19 to 0.20 migration
> ---------------------------------------------
>                 Key: HBASE-1959
>                 URL: https://issues.apache.org/jira/browse/HBASE-1959
>             Project: Hadoop HBase
>          Issue Type: New Feature
>            Reporter: Dave Latham
>         Attachments: 1959-branch0.20.patch
> Current migration has no support for compression.  You can always enable compression
after migration and it should take effect after the next major compaction.  For anyone else
who may still have a significant amount of compressed data in 0.19, this might prove useful.
> We have a large amount of data stored in 0.19 under GZip compression, at an effective
10x rate.  Out of concern for migration speed and the risk of using more capacity than our
cluster has, we didn't want to first write out all the data uncompressed before enabling LZO
> I'm attaching a small patch that allowed us to migrate the data directly to a compressed
table format.  To use it, add a migrate.compression property to hbase-site.xml with a value
of LZO or GZ, and it should compress all tables during migration.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message