hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce
Date Tue, 19 Jan 2016 02:28:40 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15106139#comment-15106139

Junping Du commented on MAPREDUCE-6608:

bq. I am just thinking that will it be ended up having too many small files in hdfs if we
are planning to store AM information in HDFS.
It shouldn't be a problem if we cleanup this file when job get completed (like hookup in job
commit stage).

[~srikanth.sampath], I think there are still many details need to be taken care besides new
attempt AM address. I noticed this JIRA hasn't been updated for any patch work for a while.
Is this work a high priority for you in the short term? If not, do you mind I take it and
move it forward? Thanks!

> Work Preserving AM Restart for MapReduce
> ----------------------------------------
>                 Key: MAPREDUCE-6608
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6608
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Srikanth Sampath
>            Assignee: Srikanth Sampath
>         Attachments: WorkPreservingMRAppMaster.pdf
> Providing a framework for work preserving AM is achieved in [YARN-1489|https://issues.apache.org/jira/browse/YARN-1489].
 We would like to take advantage of this for MapReduce(MR) applications.  There are some challenges
which have been described in the attached document and few options discussed.  We solicit
feedback from the community.

This message was sent by Atlassian JIRA

View raw message