hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3150) Move task file promotion into the task
Date Mon, 01 Sep 2008 13:56:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Devaraj Das updated HADOOP-3150:
--------------------------------

    Status: Open  (was: Patch Available)

Some comments on the last patch:
1. Remove the check for tip.isComplete (it is not necessary) for cleanup task in JIP.failedTask
2. Make sure launchedCleanup is set to false if the cleanup task is failed or killed from
UI or commandLine or because of lost TT
3. Make sure the cleanupTip.progress is same any other tip progress.
4. Change version number for the JobSubmissionProtocol and InterTrackerProtocol because of
JobStatus and TaskStatus changes
5. Change the name of setCleanupJobTask in Task.java to setCleanupTask, because it confusing
with TIP.setCleanupTask
6. You should have a error code for the case where you fail to report commitPending to TT
or ask for commitapproval from TT.  for e.g., error 
code 67and 68,  just like if ping fails, then System.exit(65) is called.
7. In JobQueueTaskScheduler and LimitTasksPerJobScheduler, move the iteration over jobs for
cleanup tasks to the top of assignTasks, i.e. before 
calculating the loads.
8. In JobStatus, call the old constructor from new constructor by passing 0.0f for cleanupProgress.
9. The kill for commit pending tasks that should be killed should be handled in getTasksToKill
10. JIP.updateTaskStatus should update the progress of the job only when it gets the statuses
of regular tasks
11. In TaskTrackerStatus, countMapTasks/countReduceTasks should count COMMIT_PENDING tasks
too

> Move task file promotion into the task
> --------------------------------------
>
>                 Key: HADOOP-3150
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3150
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.19.0
>
>         Attachments: 3150.patch, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
patch-3150.txt, patch-3150.txt, patch-3150.txt
>
>
> We need to move the task file promotion from the JobTracker to the Task and move it down
into the output format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message