hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: Map output files are SequenceFileFormat
Date Mon, 14 Feb 2011 16:44:44 GMT

On Mon, Feb 14, 2011 at 8:51 PM, Pedro Costa <psdc1978@gmail.com> wrote:
> Hi,
> 1 - The map output files are always of the type SequenceFileFormat?

If you mean the Map-intermediate files, then no - they're IFiles.
Otherwise, if your OutputFormat is set to a SequenceFileOutputFormat,
then yes these type of files would be created.

Map-Reduce intermediate files are of the IFile format. It's not part
of the public API, but you may read its implementation in

SequenceFiles are almost similar, but are built for better K-V file
operations such as skipping over keys, etc. which is not essentially
required in case of partitioned-and-sorted-data-containing IFiles.

Harsh J

View raw message