hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-947) OutputCommitter should have an abortJob method
Date Wed, 14 Oct 2009 05:55:31 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765411#action_12765411

Arun C Murthy commented on MAPREDUCE-947:

bq. We thought of that but then for an unsuccessful job we dont expect many states other than
killed or failed. Do have a case in mind where a enum is must?

It is definitely better to be more precise with your apis... a boolean to distinguish betweeen
failed and killed is weak. Also it will ensure we don't need 2 booleans in Task, just the
current *jobCleanup* and the enum JobStatus.State.

bq. Arun, _done is ambiguous (succeeded/failed/killed). _success is precise. 

Uh, ok. I'd definitely go for all caps though  - _SUCCEEDED or something like that. It has
the happy side-effect of appearing at the top of the 'ls' output! *smile*

bq. For the task its commitTask(), abortTask() and setupTask(). For a job, cleanupJob() already
exists and I havent changed that for backward compatibility. May be we can rename cleanupJob()
to commitJob() in a new jira.

That was when I let out my sigh. We cannot do that now - too late, we just have to live with

> OutputCommitter should have an abortJob method
> ----------------------------------------------
>                 Key: MAPREDUCE-947
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-947
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>             Fix For: 0.21.0
>         Attachments: mapred-948-v1.12-branch-0.20-internal.patch, mapred-948-v1.12.patch,
mapred-948-v1.13-branch-0.20-internal.patch, mapred-948-v1.2.patch, mapred-948-v1.3.patch,
mapred-948-v1.4.patch, mapred-948-v1.7.patch, mapred-948-v2.1-branch-0.20.patch, mapred-948-v2.3-branch-0.20.patch,
> The OutputCommitter needs an abortJob method to clean up from failed jobs. Currently
there is no way to distinguish between failed or succeeded jobs, making it impossible to write
output promotion code.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message