hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-2705) tasks localized and launched serially by TaskLauncher - causing other tasks to be delayed
Date Mon, 18 Jul 2011 19:41:57 GMT
tasks localized and launched serially by TaskLauncher - causing other tasks to be delayed
-----------------------------------------------------------------------------------------

                 Key: MAPREDUCE-2705
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2705
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: tasktracker
    Affects Versions: 0.20.205.0
            Reporter: Thomas Graves
            Assignee: Thomas Graves


The current TaskLauncher serially launches new tasks one at a time. During the launch it does
the localization and then starts the map/reduce task.  This can cause any other tasks to be
blocked waiting for the current task to be localized and started. In some instances we have
seen a task that has a large file to localize (1.2MB) block another task for about 40 minutes.
This particular task being blocked was a cleanup task which caused the job to be delayed finishing
for the 40 minutes.


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message