hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Spreitzer <mspre...@us.ibm.com>
Subject Re: output from one map reduce job as the input to another map reduce job?
Date Tue, 27 Sep 2011 20:36:59 GMT
It looks to me like Oozie will not do what was asked.  In 
http://yahoo.github.com/oozie/releases/3.0.0/WorkflowFunctionalSpec.html#a0_Definitions 
I see:

3.2.2 Map-Reduce Action
...
The workflow job will wait until the Hadoop map/reduce job completes 
before continuing to the next action in the workflow execution path.

That implies to me that the output of one job is held in some intermediate 
storage (likely HDFS) for a while before being read by the consuming 
job(s).

Regards,
Mike Spreitzer
Mime
View raw message