hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Bockelman <bbock...@cse.unl.edu>
Subject Re: Regarding Job tracker
Date Wed, 28 Apr 2010 18:27:45 GMT
Interesting!  Here's what the Condor folks have been doing with MapReduce:

http://www.cs.wisc.edu/condor/CondorWeek2010/condor-presentations/thain-condor-hadoop.pdf

Dunno why we don't see more of them (maybe it's just because I'm not subscribed to the MAPREDUCE
mailing list?  I have too many emails...).

Oh well... my point was that one can have multiple schedulers on a single cluster, have great
utilization, and increase your reliability.  It has its disadvantages too - multiple schedulers
while preserving data locality is tough, it's "yet another component" for small sites.

Brian

On Apr 28, 2010, at 12:49 PM, Arun C Murthy wrote:

> 
>> They have gotten lots of mileage out of breaking the scheduling and the resource
provision into two different components.  Having multiple jobtrackers would be very advantageous
if it didn't require you to partition your pool.
>> 
> 
> https://issues.apache.org/jira/browse/MAPREDUCE-279
> 
> Arun


Mime
View raw message