hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Platform MapReduce - Enterprise Features
Date Tue, 13 Sep 2011 11:48:20 GMT
On 12/09/11 19:04, Chi Chan wrote:
> Are any Hadoop implementations planning to add "enterprise features"
> in Platform MapReduce?
> http://www.youtube.com/watch?v=QV4wJifsqbQ
> http://www.youtube.com/watch?v=cDfZTx-BOyY
> http://www.youtube.com/watch?v=MEKXo-1hnkQ
> Platform said that its MapReduce implementation totally replaces the
> JobTracker, while the rest of the Hadoop stack is unchanged. Is there
> a Hadoop API that would allow external batch systems (like Grid Engine
> or Open Grid Scheduler, PBS, Condor, SLURM, etc) to plug into Hadoop?
> --Chi

It would be nice to have a scheduler that worked with the other native 
schedulers -and have those schedulers work with Hadoop- so neither 
system will overcommit. The other schedulers aren't topology aware so 
will run code anywhere; it's Hadoop that cares more where stuff goes.

The way Hadoop is architected you could do the hadoop side of the 
scheduler without breaking anything; I don't know about the others.

View raw message