hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Venner <jason.had...@gmail.com>
Subject Re: Saving Intermediate Results from the Mapper
Date Mon, 23 Nov 2009 02:27:45 GMT
You can manually write the map output to a new file, there are a number of
examples of opening a sequence file and writing to it on the web or in the
example code for various hadoop books.

You can also disable the removal of intermediate data, which will result in
potentially large amounts of data being left in the mapred.local.dir.

On Sun, Nov 22, 2009 at 3:56 PM, Gordon Linoff <glinoff@gmail.com> wrote:

> I am starting to learn Hadoop, using the Yahoo virtual machine with version
> 0.18.
> My question is rather simple.  I would like to execute a map/reduce job.
>  In
> addition to getting the results from the reduce, I would also like to save
> the intermediate results from the map in another HDFS file.  Is this
> possible?
> --gordon

Pro Hadoop, a book to guide you from beginner to hadoop mastery,
www.prohadoopbook.com a community for Hadoop Professionals

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message