hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: speculative execution in yarn
Date Wed, 17 Oct 2012 18:28:03 GMT
Speculative execution is a per-job concept, so in 2.* release line, it is MR AM's responsibility.

Because there is a two level scheduling - one at RM and one at AM, AMs have not way of figuring
out if there are other jobs are not. In general, AMs send container requests in a greedy manner
and leave the responsibility of cross-job fairness to the RM and the scheduler.

In sum, to answer your question, speculative execution will be triggered only based on task
run times, and irrespective of other jobs.

Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/

On Oct 17, 2012, at 5:19 AM, Radim Kolar wrote:

> Is yarn smart enough to do speculative execution only if there are no other jobs running
or queued?


Mime
View raw message