hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raju Bairishetti (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce
Date Tue, 19 Jan 2016 01:35:40 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15106086#comment-15106086
] 

Raju Bairishetti commented on MAPREDUCE-6608:
---------------------------------------------

I agree that storing state in zookeeper may have scalability issues. I am just thinking that
will it be ended up having too many small files in hdfs if we are planning to store AM information
in HDFS.

> Work Preserving AM Restart for MapReduce
> ----------------------------------------
>
>                 Key: MAPREDUCE-6608
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6608
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Srikanth Sampath
>            Assignee: Srikanth Sampath
>         Attachments: WorkPreservingMRAppMaster.pdf
>
>
> Providing a framework for work preserving AM is achieved in [YARN-1489|https://issues.apache.org/jira/browse/YARN-1489].
 We would like to take advantage of this for MapReduce(MR) applications.  There are some challenges
which have been described in the attached document and few options discussed.  We solicit
feedback from the community.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message