hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Chansler (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3150) Move task file promotion into the task
Date Tue, 21 Oct 2008 19:46:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Robert Chansler updated HADOOP-3150:
------------------------------------

    Release Note: 
Moved task file promotion from to the Task. When the task has finished, it will do a commit
and is declared SUCCEDED. Job cleanup is done by a separate task. Job is declared SUCCEDED/FAILED
after the cleanup task has finished. Added public classes org.apache.hadoop.mapred.JobContext,
TaskAttemptContext, OutputCommitter and FileOutputCommiitter. Added public APIs:   public
OutputCommitter getOutputCommitter() and 
public void setOutputCommitter(Class<? extends OutputCommitter> theClass) in org.apache.hadoop.mapred.JobConf

  was:
1. Moved the task file promotion from the JobTracker to the Task and moved it down into the
OutputCommitter. Now, once the task has finished executing its map/reduce code, it will do
a commit if required. Then the task is declared SUCCEDED.
2. Job cleanup is done by a separate task when all maps and reduces are completed or if the
job is killed. Job is declared SUCCEDED/FAILED after the cleanup task has finished. 
3. Added public classes org.apache.hadoop.mapred.JobContext, org.apache.hadoop.mapred.TaskAttemptContext,
org.apache.hadoop.mapred.OutputCommitter and org.apache.hadoop.mapred.FileOutputCommiitter
4. Added public apis:   public OutputCommitter getOutputCommitter() and 
public void setOutputCommitter(Class<? extends OutputCommitter> theClass) in org.apache.hadoop.mapred.JobConf


> Move task file promotion into the task
> --------------------------------------
>
>                 Key: HADOOP-3150
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3150
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.19.0
>
>         Attachments: 3150 test plan.txt, 3150.patch, patch-3150.txt, patch-3150.txt,
patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
patch-3150.txt, patch-3150.txt, patch-3150.txt
>
>
> We need to move the task file promotion from the JobTracker to the Task and move it down
into the output format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message