hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-2702) [MR-279] OutputCommitter changes for MR Application Master recovery
Date Wed, 05 Oct 2011 05:04:34 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Arun C Murthy updated MAPREDUCE-2702:

    Attachment: MAPREDUCE-2702.patch

Updated patch, I've incorporated f/b on better docs for the api.

Regarding return value v/s exception, it's odd to throw an exception when it's expected that
many impls won't deal with it - a default impl of throwing an exception is odd too - I've
kept it as-is.

I've changed names of private apis from get(Job,Task)TempDirName to get(Job,Task)AttemptBaseDirName.

As Sharad suggested, I'll open a new jira for the old api.
> [MR-279] OutputCommitter changes for MR Application Master recovery
> -------------------------------------------------------------------
>                 Key: MAPREDUCE-2702
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2702
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: mrv2
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>            Priority: Blocker
>         Attachments: MAPREDUCE-2702.patch, MAPREDUCE-2702.patch, mr2702_v1.patch, mr2702_v2.patch,
mr2702_v3.patch, mr2702_v4.patch
> In MR AM recovers from a crash, it only reruns the non completed tasks. The completed
tasks (along with their output, if any) needs to be recovered from the previous life. This
would require some changes in OutputCommitter.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message