hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-1398) TaskLauncher remains stuck on tasks waiting for free nodes even if task is killed.
Date Fri, 22 Jan 2010 17:09:21 GMT
TaskLauncher remains stuck on tasks waiting for free nodes even if task is killed.
----------------------------------------------------------------------------------

                 Key: MAPREDUCE-1398
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1398
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: tasktracker
            Reporter: Hemanth Yamijala


Tasks could be assigned to trackers for slots that are running other tasks in a commit pending
state. This is an optimization done to pipeline task assignment and launch. When the task
reaches the tracker, it waits until sufficient slots become free for it. This wait is done
in the TaskLauncher thread. Now, while waiting, if the task is killed externally (maybe because
the job finishes, etc), the TaskLauncher is not notified of this. So, it continues to wait
for the killed task to get sufficient slots. If slots do not become free for a long time,
this would result in considerable delay in waking up the TaskLauncher thread. If the waiting
task happens to be a high RAM task, then it is also wasteful, because by waking up, it can
make way for normal tasks that can run on the available number of slots.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message