hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer <awittena...@linkedin.com>
Subject Re: Do we shoot ourselves by using all task slots?
Date Fri, 28 May 2010 18:54:48 GMT

On May 28, 2010, at 11:43 AM, Todd Lipcon wrote:

> Hi Allen,
> Recent versions of the fair scheduler have configurations for "delay scheduling" - essentially,
it will wait for a few seconds when a slot opens up to try to find a local task before assigning
a non-local one. This is specifically to avoid the issue you're describing.
> Check out Matei's Eurosys 2010 paper here:
> http://www.cs.berkeley.edu/~matei/papers/2010/eurosys_delay_scheduling.pdf
> I believe this got lumped in with MAPREDUCE-706.

Thanks.  I'll take a look at this and see how applicable this is to Capacity.  
View raw message