hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: Map output files are SequenceFileFormat
Date Mon, 14 Feb 2011 16:44:44 GMT
Hello,

On Mon, Feb 14, 2011 at 8:51 PM, Pedro Costa <psdc1978@gmail.com> wrote:
> Hi,
>
> 1 - The map output files are always of the type SequenceFileFormat?

If you mean the Map-intermediate files, then no - they're IFiles.
Otherwise, if your OutputFormat is set to a SequenceFileOutputFormat,
then yes these type of files would be created.

Map-Reduce intermediate files are of the IFile format. It's not part
of the public API, but you may read its implementation in
src/java/org/apache/hadoop/mapred/IFile.java.

SequenceFiles are almost similar, but are built for better K-V file
operations such as skipping over keys, etc. which is not essentially
required in case of partitioned-and-sorted-data-containing IFiles.

-- 
Harsh J
www.harshj.com

Mime
View raw message