hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2705) tasks localized and launched serially by TaskLauncher - causing other tasks to be delayed
Date Mon, 01 Aug 2011 21:44:48 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073805#comment-13073805

Thomas Graves commented on MAPREDUCE-2705:

when I first submitted the patch the tests on trunk were already broken. For instance see
https://builds.apache.org/view/G-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/472/ which is
for a different jira.  Let me rerun now just to double check and I'll update you on results.

> tasks localized and launched serially by TaskLauncher - causing other tasks to be delayed
> -----------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-2705
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2705
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions:
>            Reporter: Thomas Graves
>            Assignee: Thomas Graves
>             Fix For:, 0.23.0
>         Attachments: MAPREDUCE-2705-branch20.patch, MAPREDUCE-2705-trunk.patch
> The current TaskLauncher serially launches new tasks one at a time. During the launch
it does the localization and then starts the map/reduce task.  This can cause any other tasks
to be blocked waiting for the current task to be localized and started. In some instances
we have seen a task that has a large file to localize (1.2MB) block another task for about
40 minutes. This particular task being blocked was a cleanup task which caused the job to
be delayed finishing for the 40 minutes.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message