hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jothi Padmanabhan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-947) OutputCommitter should have an abortJob method
Date Fri, 09 Oct 2009 09:30:31 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jothi Padmanabhan updated MAPREDUCE-947:
----------------------------------------

    Status: Open  (was: Patch Available)

A few comments:

# The parameter should read mapreduce.fileoutputcommitter.marksuccessfuljobs
# You should declare a static string for this configuraiton and use that in the code, as is
being done for other configs
# The default value for the config should be false (to be backward compatible)
# Do the following methods in FileOutputCommitter need to be static? markSuccessfulOutputDir,
cleanup
# The JavaDoc for abortJob method should document what the boolean is doing (in the mapreduce.outputcommitter)
# Minor Nit. In the test case, the comment for MyMapper (Throws OOM for the first attempt)
is wrong, MyMapper throws OOM always
# Some lines in test case is exceeding 80 characters
# Using a delaysetupcommitter in the test case for checking absence of output directory looks
wrong, cleanup is called only if the setup succeeds. Instead the test case should actually
use a delayed mapper.

> OutputCommitter should have an abortJob method
> ----------------------------------------------
>
>                 Key: MAPREDUCE-947
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-947
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>             Fix For: 0.21.0
>
>         Attachments: mapred-948-v1.12-branch-0.20-internal.patch, mapred-948-v1.12.patch,
mapred-948-v1.2.patch, mapred-948-v1.3.patch, mapred-948-v1.4.patch, mapred-948-v1.7.patch,
mapred-948-v2.1-branch-0.20.patch
>
>
> The OutputCommitter needs an abortJob method to clean up from failed jobs. Currently
there is no way to distinguish between failed or succeeded jobs, making it impossible to write
output promotion code.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message