hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-947) OutputCommitter should have an abortJob method
Date Tue, 20 Oct 2009 06:44:59 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arun C Murthy updated MAPREDUCE-947:
------------------------------------

    Status: Open  (was: Patch Available)

Jothi, this is looking good... 

I have some minor nits I should have pointed out earlier, my bad - apologies:

# FileOutputCommitter shouldn't override deprecated cleanupJob at all.
# FileOutputCommitter silently ignores cases where FileOutputFormat.getOutputPath returns
null. This isn't new - I just thought we should fix it to log a warning alongwith context
(i.e. which function) where it returns null in all cases.
# Task.java should just call the final org.apache.hadoop.mapred.OutputCommitter.abortJob(JobContext,
State) rather than call 
org.apache.hadoop.mapred.OutputCommitter.abortJob(JobContext, int) after type-conversion from
State to int
# JobStatus.getOldNewJobRunState can just use org.apache.hadoop.mapreduce.JobStatus.State.getValue(),
no?
# javadoc for o.a.h.mapred.OutputCommitter.cleanupJob needs a space between the link and *instead*
# javadoc for o.a.h.{mapred|mapreduce}.OutputCommitter.abortJob can be improved to use links
for the FAILED/KILLED states as {@link JobStatus#FAILED} etc.


> OutputCommitter should have an abortJob method
> ----------------------------------------------
>
>                 Key: MAPREDUCE-947
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-947
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 0.21.0
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>             Fix For: 0.22.0
>
>         Attachments: mapred-948-v1.12-branch-0.20-internal.patch, mapred-948-v1.12.patch,
mapred-948-v1.13-branch-0.20-internal.patch, mapred-948-v1.2.patch, mapred-948-v1.3.patch,
mapred-948-v1.4.patch, mapred-948-v1.7.patch, mapred-948-v2.1-branch-0.20.patch, mapred-948-v2.3-branch-0.20.patch,
mapred-948-v2.3.patch, mapred-948-v3.1.patch, mapred-948-v3.2.patch, mapred-948-v3.4.patch,
mr-947-trunk.patch, mr-947-trunk.patch
>
>
> The OutputCommitter needs an abortJob method to clean up from failed jobs. Currently
there is no way to distinguish between failed or succeeded jobs, making it impossible to write
output promotion code.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message