hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eason.Lee" <leongf...@gmail.com>
Subject Re: do all mappers finish before reducer starts
Date Wed, 27 Jan 2010 02:55:55 GMT
No,Reduce will start as soon as Map starts
So reduce can start transfer map outputs to local when some of they are
finished

2010/1/27 adeelmahmood <adeelmahmood@gmail.com>

>
> I just have a conceptual question. My understanding is that all the mappers
> have to complete their job for the reducers to start working because
> mappers
> dont know about each other so we need values for a given key from all the
> different mappers so we have to wait until all mappers have collectively
> given the system all possible values for a key .so that then that can be
> passed on the reducer ..
> but when I ran these jobs .. almost everytime before the mappers are all
> done the reducers start working .. so it would say map 60% reduce 30% ..
> how
> does this works
> Does it finds all possibly values for a single key from all mappers .. pass
> that on the reducer and then works on other keys
> any help is appreciated
> --
> View this message in context:
> http://old.nabble.com/do-all-mappers-finish-before-reducer-starts-tp27330927p27330927.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message