hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrzej Bialecki ...@getopt.org>
Subject Re: Job scheduling (Re: Unable to run more than one job concurrently)
Date Fri, 19 May 2006 19:56:00 GMT
Paul Sutter wrote:
> priority vs pretend dates: whatever is most trivial, i'm in favor. we 
> could
> certainly use it today.
>
> delaying reducers: regardless of the bottleneck in the system, andrzej's
> problem was that reducers were sitting idle while the mappers of another
> task were running.

Exactly. There is no point in starting any reducers if we haven't even 
scheduled a single map task for the job. As soon as at least one map 
task is scheduled, I'm fine with starting reducers, I agree with 
arguments about network bottlenecks that Doug presented.

-- 
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com



Mime
View raw message