hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: speculative execution in yarn
Date Wed, 17 Oct 2012 18:28:03 GMT
Speculative execution is a per-job concept, so in 2.* release line, it is MR AM's responsibility.

Because there is a two level scheduling - one at RM and one at AM, AMs have not way of figuring
out if there are other jobs are not. In general, AMs send container requests in a greedy manner
and leave the responsibility of cross-job fairness to the RM and the scheduler.

In sum, to answer your question, speculative execution will be triggered only based on task
run times, and irrespective of other jobs.

+Vinod Kumar Vavilapalli
Hortonworks Inc.

On Oct 17, 2012, at 5:19 AM, Radim Kolar wrote:

> Is yarn smart enough to do speculative execution only if there are no other jobs running
or queued?

View raw message