hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Cuffney <cuffn...@gmail.com>
Subject Job Controller for MapReduce task assignment
Date Fri, 07 Sep 2012 07:54:38 GMT

Which class handles the top level partitioning for MapReduce?  It's
possible I have a misunderstanding of how this is handled, but in my view,
there is a top level controller which kicks off the whole process; it
handles partitioning of the input and distribution of the input segments to
the various machines/tasks.  I have been searching through a lot of the Job
classes, and they all seem to handle a single task, whereas it is important
for me to perform some work at the highest level controller, if that
exists.  Any info on what I'm looking for/if I'm on the wrong track would
be much appreciated.

Thanks for the help,

View raw message