hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars George <lars.geo...@gmail.com>
Subject Re: Compress output using HFileOutputFormat
Date Thu, 27 Jan 2011 06:46:51 GMT
Hi Nanheng,

You have to use its own configuration key to enable it. So when you
create the job configuration do add an

conf.set("hfile.compression", "gz");

Obviously before you create the job or use

job.getConfiguration().set("hfile.compression", "gz");



On Thu, Jan 27, 2011 at 2:25 AM, Nanheng Wu <nanhengwu@gmail.com> wrote:
> I am sorry if this has been asked before: To bulk load into HBase I am
> using a mapper only job to generate the HFiles and then run
> loadtable.rb. Everything seems fine now but I want to turn on GZIP
> compression on the table. I did
> HFileOutputFormat.setCompressOutput(job, true); in the MR job and set
> Compression to "GZ" on the column family I want. But the compression
> doesn't seem to work, the final table size is same as the
> non-compressed verison. What am I missing? I'd really appreciate any
> help. Thanks!

View raw message