hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ioan Eugen Stan <stan.ieu...@gmail.com>
Subject Re: LZO with sequenceFile
Date Sun, 26 Feb 2012 16:25:34 GMT
2012/2/26 Mohit Anchlia <mohitanchlia@gmail.com>:
> Thanks. Does it mean LZO is not installed by default? How can I install LZO?

The LZO library is released under GPL and I believe it can't be
included in most distributions of Hadoop because of this (can't mix
GPL with non GPL stuff). It should be easily available though.

> On Sat, Feb 25, 2012 at 6:27 PM, Shi Yu <shiyu@uchicago.edu> wrote:
>
>> Yes, it is supported by Hadoop sequence file. It is splittable
>> by default. If you have installed and specified LZO correctly,
>> use these:
>>
>>
>> org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputForma
>> t.setCompressOutput(job,true);
>>
>> org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputForma
>> t.setOutputCompressorClass(job,com.hadoop.compression.lzo.LzoC
>> odec.class);
>>
>> org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputForma
>> t.setOutputCompressionType(job,
>> SequenceFile.CompressionType.BLOCK);
>>
>> job.setOutputFormatClass(org.apache.hadoop.mapreduce.lib.outpu
>> t.SequenceFileOutputFormat.class);
>>
>>
>> Shi
>>



-- 
Ioan Eugen Stan
http://ieugen.blogspot.com/

Mime
View raw message