hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amogh Vasekar <am...@yahoo-inc.com>
Subject Re: Pass the TaskId from map to Reduce
Date Thu, 18 Feb 2010 19:51:01 GMT
Hi Ankit,
>>however the the issue that i am facing that I was expecting all the maps to finish
before any reduce starts.
This is exactly how it happens, reducers poll map tasks for data and begin user code only
after all maps complete.

>>when is closed function called after every map or after all the maps?
Once for every map task, after all the input data for that task is consumed.

Also, can you let us know where you are writing your data, ie in cwd of the task or directly
on hdfs?

Thanks,
Amogh

On 2/18/10 8:58 AM, "ANKITBHATNAGAR" <abhatnagar@vantage.com> wrote:



Hi Don,
Thanks for your reply.
I already tried this approach, however the the issue that i am facing that I
was expecting all the maps to finish before any reduce starts.This is not
happening for me.
It looks like as one map finishes reduce starts.
Thats why I called close().?
Could you tell me when is closed function called after every map or after
all the maps?

Am I doing something wrong?


Thanks
Ankit
--
View this message in context: http://old.nabble.com/Pass-the-TaskId-from-map-to-Reduce-tp27575531p27634001.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message