hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-947) OutputCommitter should have an abortJob method
Date Wed, 14 Oct 2009 05:12:31 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765396#action_12765396
] 

Amar Kamat commented on MAPREDUCE-947:
--------------------------------------

bq. I wish OutputCommitter's methods were called commit and abort rather than the current
apis of cleanupJob and abortJob... sigh!
This is insync with the apis for tasks i.e abortTask() and cleanupTask()

bq. OutputCommitter.abortJob should take a JobStatus.State enum rather than a boolean for
jobFailed/jobKilled
We thought of that but then for an unsuccessful job we dont expect many states other than
killed or failed. Do have a case in mind where a enum is must?

bq. I'd probably set SUCCEEDED_FILE_NAME to be _done or _DONE
Arun, _done is ambiguous (succeeded/failed/killed). _success is precise. 

bq. We adding a findbugs suppression, I assume this is for the JobContext being from the different
package? We should justify them in future. 
Yes. The findbugs is confused between the two classes (mapred and mapreduce). 

bq. Default value of mapreduce.fileoutputcommitter.marksuccessfuljobs should be 'true' and
we shouldn't put it in mapred-default.xml; it should be a hidden parameter so we can remove
this in the next release.
It is true and its not mentioned in mapred-default. I have made the default value false for
testcases so that they remain unchanged. 

> OutputCommitter should have an abortJob method
> ----------------------------------------------
>
>                 Key: MAPREDUCE-947
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-947
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>             Fix For: 0.21.0
>
>         Attachments: mapred-948-v1.12-branch-0.20-internal.patch, mapred-948-v1.12.patch,
mapred-948-v1.13-branch-0.20-internal.patch, mapred-948-v1.2.patch, mapred-948-v1.3.patch,
mapred-948-v1.4.patch, mapred-948-v1.7.patch, mapred-948-v2.1-branch-0.20.patch, mapred-948-v2.3-branch-0.20.patch,
mapred-948-v2.3.patch
>
>
> The OutputCommitter needs an abortJob method to clean up from failed jobs. Currently
there is no way to distinguish between failed or succeeded jobs, making it impossible to write
output promotion code.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message