hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 李钰 <car...@gmail.com>
Subject Re: Reduce starts as Map finishes
Date Mon, 21 Jun 2010 03:00:54 GMT
Hi Eric,

Here I've got a question: why the
mapred.reduce.slowstart.completed.maps parameter is added in 0.20.x release?
What's the designing thought? I found sometimes setting this parameter do
improve performance, but not quite clear about why. Do you know under what
conditions should this parameter be used and play a role? Thanks and looking
forward to your reply.

Best Regards,
Carp

2010/6/21 Eric Sammer <esammer@cloudera.com>

> The reducers will start the copy phase while mappers are still running
> but they will not call the user reduce method until all mappers are
> complete. This is an optimization that shouldn't matter to the user.
>
> Eric
>
> On Jun 20, 2010, at 8:11 AM, biblio84 <fariha.atta.pk@gmail.com> wrote:
>
> >
> > Same is the problem with me. Why does reducer start working before
> > mappers
> > are done? Have u found the answer?
> > --
> > View this message in context:
> http://old.nabble.com/Reduce-starts-as-Map-finishes-tp27662411p28939648.html
> > Sent from the Hadoop core-user mailing list archive at Nabble.com.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message