hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-5718) MR AM should tolerate RM restart/failover during commit
Date Mon, 13 Jan 2014 19:30:53 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vinod Kumar Vavilapalli updated MAPREDUCE-5718:

    Priority: Major  (was: Blocker)

This was by design to handle racing AMs in the presence of partitions. Reducing priority.

This will go away once we have work-preserving RM restart.

For the shorter term, I think it makes sense to have the AM delete the file before failing.

> MR AM should tolerate RM restart/failover during commit
> -------------------------------------------------------
>                 Key: MAPREDUCE-5718
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5718
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mr-am
>    Affects Versions: 2.4.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>              Labels: ha
> While testing RM HA, we ran into this issue where if the RM fails over while an MR AM
is in the middle of a commit, the subsequent AM gets spawned but dies with a diagnostic message
- "We crashed durring a commit". 

This message was sent by Atlassian JIRA

View raw message