hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tao Xiao <xiaotao.cs....@gmail.com>
Subject Re: How does mapreduce job determine the compress codec
Date Sat, 14 Dec 2013 05:16:35 GMT
I suggest you download the lzo compressed file, no matter weather it has a
lzo extension as its file name,  and open it in the form of hex bytes with
tools like UltraEdit, and have a look at its heading contents.

2013/12/14 Jiayu Ji <jiayu.ji@gmail.com>

> Hi
> I am having this question on how does mapreduce job determine the compress
> codec on hdfs. From what I read on the definitive guide (page 86)," the
> CompressionCodecFactory provides a way of mapping a filename extension to a
> CompressionCodec using its getCodec() method". I did a test with a lzo
> compressed file without a lzo extension. However, the mapreduce job was
> still able to get the right codec. Does anyone know why? Thanks in advance.
> Jiayu

View raw message