hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: What happens after COMMIT_PENDING?
Date Sat, 12 Mar 2011 06:33:44 GMT

On Wed, Mar 9, 2011 at 3:17 PM, Pedro Costa <psdc1978@gmail.com> wrote:
> I know that when a task is in COMMIT_PENDING, is requesting something
> to the TaskTracker or the JobTracker, for the task become SUCCESSFUL.
> But, in detail, I don't know nothing. Can anyone explain in more
> details what's happening?

Every task sets its state to COMMIT_PENDING upon completion and
requests for a 'commit' status from the JobTracker. If the JT decides
that this task instance may be given a OK-to-commit, it does, else it
kills it. This is useful in speculative execution where multiple tasks
of the same ID run in parallel (only one ought to be committed, the
other(s) killed).

See JobTracker.updateTaskStatuses() and finishing-up parts of the
JobTracker.heartbeat() call that deals with the saving and killing
task actions, to see how COMMIT_PENDING is utilized.

Harsh J

View raw message