hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5165) getSetupAndCleanupTasks should return multiple tasks in a heartbeat
Date Mon, 16 Feb 2009 12:43:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673867#action_12673867
] 

Vinod K V commented on HADOOP-5165:
-----------------------------------

Some observations:
 - There are now six iterations(!!) over the list of all jobs for getting setup and cleanup
tasks. The code can be re-factored to get this count down to three, for e.g., by combining
JobInProgress.obtainJobCleanupTask with map and reduce slots into a single loop.
 - With this patch, even after the above refactoring, we go through the list of jobs multiple
times in all cases while without this patch, we do it only in the worst case. Though this
may quicken the launching of job-setup, job-cleanup and task-cleanup tasks, it will affect
all the heartbeats. I am just curious how the grid-mix numbers have turned out with this patch.

> getSetupAndCleanupTasks should return multiple tasks in a heartbeat
> -------------------------------------------------------------------
>
>                 Key: HADOOP-5165
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5165
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.20.0, 0.21.0
>
>         Attachments: patch-5165-1.txt, patch-5165.txt
>
>
> getSetupAndCleanupTasks in JobTracker returns one task per heartbeat. With HADOOP-3136,
schedulers give multiple tasks  per heartbeat, getSetupAndCleanupTasks should also give multiple
tasks per heartbeat.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message