hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6726) YARN Registry based AM discovery with retry and in-flight task persistent via JHS
Date Mon, 26 Sep 2016 09:04:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15522514#comment-15522514
] 

Jian He commented on MAPREDUCE-6726:
------------------------------------

[~srikanth.sampath], thanks for the patch , I looked at it.

IIUC, we are also going to have a different mechanism to retrieve the AM address via YARN-4758.
The patch right now is hardcoded to depend on registry approach only, this part of the code
 needs to be made pluggable so that the approach listed in YARN-4758 can be plugged in.  We
could implement different FailoverProvider like RegistryBasedFailoverProvider or RPCBasedFailoverProvider.


Regarding the JVMId changes, could you separate that out and upload it on to MAPREDUCE-6754
? we can get that reviewed and committed first. 


> YARN Registry based AM discovery with retry and in-flight task persistent via JHS
> ---------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6726
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6726
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: applicationmaster
>            Reporter: Junping Du
>            Assignee: Srikanth Sampath
>         Attachments: MAPREDUCE-6726-MAPREDUCE-6608.001.patch, MAPREDUCE-6726-MAPREDUCE-6608.001.patch,
MAPREDUCE-6726-MAPREDUCE-6608.002.patch, WorkPreservingMRAppMaster.pdf
>
>
> Several tasks will be achieved in this JIRA based on the demo patch in MAPREDUCE-6608:
> 1. AM discovery base on YARN register service. Could be replaced by YARN-4758 later due
to scale up issue.
> 2. Retry logic for TaskUmbilicalProtocol RPC connection
> 3. In-flight task recover after AM restart via JHS
> 4. Configuration to control the behavior compatible with previous when not enable this
feature (by default).
> All security related issues and other concerns discussed in MAPREDUCE-6608 will be addressed
in follow up JIRAs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message