hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-947) OutputCommitter should have an abortJob method
Date Mon, 05 Oct 2009 07:37:31 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762108#action_12762108
] 

Amar Kamat commented on MAPREDUCE-947:
--------------------------------------

This patch adds a abortJob() method to OutputCommitter which will be invoked for failed/killed
jobs in job-cleanup. By default (i.e FileOutputCommitter) the abort job will delete the _temporary
files in the output folder (same as cleanupJob()) and create a _abort file (empty file) in
the output folder. Also this patch modifies the cleanupJob() method of FileOutputCommitter
which will be invoked for successful jobs to create a _done file in the output folder. 

> OutputCommitter should have an abortJob method
> ----------------------------------------------
>
>                 Key: MAPREDUCE-947
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-947
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>             Fix For: 0.21.0
>
>         Attachments: mapred-948-v1.2.patch, mapred-948-v1.3.patch, mapred-948-v1.4.patch
>
>
> The OutputCommitter needs an abortJob method to clean up from failed jobs. Currently
there is no way to distinguish between failed or succeeded jobs, making it impossible to write
output promotion code.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message