hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From He Yongqiang <heyongqi...@software.ict.ac.cn>
Subject Re: DefaultCodec vs. LZO
Date Fri, 23 Oct 2009 20:27:30 GMT
I did a search, and find the below two links, hope they are helpful.
http://blog.oskarsson.nu/2009/03/hadoop-feat-lzo-save-disk-space-and.html
http://mail.jabber.org/pipermail/standards/2005-October/008768.html

I think Hadoop’s DefaultCodec is using native Zlib when possible.

Thanks,
Yongqiang
On 09-10-23 下午12:43, "Zheng Shao" <zshao@facebook.com> wrote:

> We are using Hadoop 0.20 and mapred.map.output.compression.Codec is set to
> DefaultCodec.
> We tried LZO but the performance seems very similar to DefaultCodec.
>  
> I heard of a lot of good words about LZO. So did anybody compare LZO with
> DefaultCodec?
> Is there a big difference?
>  
> We are running CentOS release 5.2 (Final).
>  
> Thanks,
> Zheng
>  
> 


Mime
View raw message