hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Seigel <ja...@tynt.com>
Subject Re: Newbie to HDFS compression
Date Fri, 25 Jun 2010 02:45:35 GMT
Cool.  Maybe we should start a page.

J
On 2010-06-24, at 8:16 PM, Harsh J wrote:

> On Fri, Jun 25, 2010 at 2:42 AM, Raymond Jennings III
> <raymondjiii@yahoo.com> wrote:
>> Oh, maybe that's what I meant :-)  I recall reading something on this mail group
that "the compression" in not included with the hadoop binary and that you have to get and
install it separately due to license incompatibilities.  Looking at the config xml files it's
not clear what I need to do.  Thanks.
>> 
> LZO Compression is the one you probably read about. Otherwise
> available CompressionCodecs are BZip2 and GZip, and you should be able
> to use those files just fine.
> 
> Something like FileOutputFormat.setCompressOutput(conf, true);
> 
> (Also look at mapred.output.compress configuration var for
> map-output-compression)
>> 
>> 
>> ----- Original Message ----
>> From: Eric Sammer <esammer@cloudera.com>
>> To: common-user@hadoop.apache.org
>> Sent: Thu, June 24, 2010 5:09:33 PM
>> Subject: Re: Newbie to HDFS compression
>> 
>> There is no file system level compression in HDFS. You can stored
>> compressed files in HDFS, however.
>> 
>> On Thu, Jun 24, 2010 at 11:26 AM, Raymond Jennings III
>> <raymondjiii@yahoo.com> wrote:
>>> Are there instructions on how to enable (which type?) of compression on hdfs?
 Does this have to be done during installation or can it be added to a running cluster?
>>> 
>>> Thanks,
>>> Ray
>>> 
>>> 
>>> 
>>> 
>> 
>> 
>> 
>> --
>> Eric Sammer
>> twitter: esammer
>> data: www.cloudera.com
>> 
>> 
>> 
>> 
>> 
> 
> 
> 
> -- 
> Harsh J
> www.harshj.com


Mime
View raw message