hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhuweimin" <xim-...@tsm.kddilabs.jp>
Subject RE: Did Hadoop support gz/zip format file?
Date Mon, 01 Dec 2008 03:58:53 GMT

I have a requirement of use file with the extension .Z(the file is created
by UNIX compress command) in Hadoop.
It will too be automatically recognized / handled ?

Any suggestion on how to handle the .Z file in the map task?

Best Regards


-----Original Message-----
From: Abdul Qadeer [mailto:qadeer.qadeer@gmail.com] 
Sent: Monday, November 03, 2008 4:19 PM
To: core-user@hadoop.apache.org
Subject: Re: Did Hadoop support gz/zip format file?

Hadoop supports gzip format by means of its gzip codecs.
If you meant to use default input format / record reader of
Hadoop, it will automatically recognize / handle your gzip
input files.  And if you are using some other input formats / record
readers, you can use the gzip codec provided by Hadoop to
de-compress the data and then interpret it in your record reader.

On Sun, Nov 2, 2008 at 7:49 PM, howardh <howardh@copacast.com> wrote:

> Hello,
> I have a requirement of use gz/zip format in Hadoop,   After these days
> research and learning,   It seems Hadoop didn't support gz/zip file yet,
>  is that true?
> Like I'm going to create file with gz format,  and read it later by
> FileSystem interface.     Does it feasible?     Experts!  Could you show
> some advices?
> Best Regards
> 2008-11-03
> howardh

View raw message