Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Wed, 10 Aug 2011 01:27:40 +0000 (UTC)
From: "Hudson (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <1813775060.22417.1312939660765.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <359578008.1267.1311018117850.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (MAPREDUCE-2705) tasks localized and launched
 serially by TaskLauncher - causing other tasks to be delayed
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082055#comment-13082055 ] 

Hudson commented on MAPREDUCE-2705:
-----------------------------------

Integrated in Hadoop-Mapreduce-trunk-Commit #760 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/760/])
    MAPREDUCE-2705. Permits parallel multiple task launches. Contributed by Thomas Graves.
MAPREDUCE-2705. Permits parallel multiple task launches. Contributed by Thomas Graves.

ddas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1153720
Files : 
* /hadoop/common/trunk/mapreduce/src/test/mapred/org/apache/hadoop/mapred/TestTaskLauncherThreaded.java

ddas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1153717
Files : 
* /hadoop/common/trunk/mapreduce/src/java/org/apache/hadoop/mapred/TaskTracker.java
* /hadoop/common/trunk/mapreduce/CHANGES.txt


> tasks localized and launched serially by TaskLauncher - causing other tasks to be delayed
> -----------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2705
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2705
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.20.205.0
>            Reporter: Thomas Graves
>            Assignee: Thomas Graves
>             Fix For: 0.20.205.0, 0.23.0
>
>         Attachments: MAPREDUCE-2705-branch20.patch, MAPREDUCE-2705-trunk.patch
>
>
> The current TaskLauncher serially launches new tasks one at a time. During the launch it does the localization and then starts the map/reduce task.  This can cause any other tasks to be blocked waiting for the current task to be localized and started. In some instances we have seen a task that has a large file to localize (1.2MB) block another task for about 40 minutes. This particular task being blocked was a cleanup task which caused the job to be delayed finishing for the 40 minutes.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira