hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-947) OutputCommitter should have an abortJob method
Date Wed, 14 Oct 2009 04:56:31 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765393#action_12765393
] 

Arun C Murthy commented on MAPREDUCE-947:
-----------------------------------------

Some comments:

# I wish OutputCommitter's methods were called *commit* and *abort* rather than the current
apis of *cleanupJob* and *abortJob*... sigh!
# OutputCommitter.abortJob should take a JobStatus.State enum rather than a boolean for jobFailed/jobKilled
# Similarly Task should just use the existing boolean jobCleanup and add a JobStatus.State
field which should be one of SUCCEEDED, FAILED, KILLED
# I'd probably set SUCCEEDED_FILE_NAME to be *_done* or *_DONE*
# We adding a findbugs suppression, I assume this is for the JobContext being from the different
package? We should justify them in future.
# Default value of mapreduce.fileoutputcommitter.marksuccessfuljobs should be 'true' and we
shouldn't put it in mapred-default.xml; it should be a hidden parameter so we can remove this
in the next release.


> OutputCommitter should have an abortJob method
> ----------------------------------------------
>
>                 Key: MAPREDUCE-947
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-947
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>             Fix For: 0.21.0
>
>         Attachments: mapred-948-v1.12-branch-0.20-internal.patch, mapred-948-v1.12.patch,
mapred-948-v1.13-branch-0.20-internal.patch, mapred-948-v1.2.patch, mapred-948-v1.3.patch,
mapred-948-v1.4.patch, mapred-948-v1.7.patch, mapred-948-v2.1-branch-0.20.patch, mapred-948-v2.3-branch-0.20.patch,
mapred-948-v2.3.patch
>
>
> The OutputCommitter needs an abortJob method to clean up from failed jobs. Currently
there is no way to distinguish between failed or succeeded jobs, making it impossible to write
output promotion code.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message