hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Status on MAPREDUCE-279 and plans forward
Date Fri, 10 Jun 2011 19:44:35 GMT
Another heads up.

Things are going well, we have done scale testing (350 nodes) and  
continue to do more.

We think we are nearly ready to merge MR-279 into trunk (yaay!).

A proposal for plan forward:

1. I'd like to get MAPREDUCE-2400 committed to both trunk and MR-279.  
This is a very important patch by Tom which allows to submit jobs to  
both JT and the new RM - thanks Tom!
2. Check-in the 'yarn' subfolder which is just the ResourceManager and  
NodeManager i.e. compute fabric to trunk.
3. Rework the src tree in trunk to look like the MR-279 branch to  
ensure no changes to trunk are lost - this will ensure separation  
between the 'classic' framework (JT/TT) and the MR runtime (map/reduce/ 
sort/shuffle etc.)
4. Re-play the 4-5 patches needed to the MR runtime from the MR-279  
branch.
5. Change build scripts to reflect the new reality.

In terms of timelines, I'm hoping we can get this done in the coming  
2-3 weeks.

Thoughts?

thanks,
Arun

Mime
View raw message