hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Omkar Vinit Joshi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-435) Make it easier to access cluster topology information in an AM
Date Wed, 29 May 2013 18:52:23 GMT

    [ https://issues.apache.org/jira/browse/YARN-435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13669570#comment-13669570
] 

Omkar Vinit Joshi commented on YARN-435:
----------------------------------------

After discussion with [~vinodkv] yesterday.. Here are the possible solutions to this
* At the time of application master registration along with AMToken also issue ClientDelegationToken
or may be on request in AMRMProtocol.
** Advantages: -AM doesn't need to request token to get additional information (no kerberos
authentication for secure case).
** Disadvantages: -
*** AM has to open 2 connections and manage 2 tokens (AMToken and ClientDelegationToken)
*** AM can now do all the activities which earlier only client was allowed to do (getAllApps,
forceKillApp, submitApp)

* Create a new interface something like, ClusterInfo and add ClusterNodes and ClusterMatrics
and similar info to it. Let ClientRMProtocol and AMRMProtocol extend this.
** Advantages: -
*** AM has to create only one connection
*** AM doesn't get by default or on request ClientDelegationToken (for secure env.). So AM
is not allowed to do app activities on ClientRMProtocol (submitApp, killApp, getAppStatus)
unless Client itself share this token with AM.
*** Connection management will be very simple for AM and will get all required info.
** Disadvantages: -
*** AMRMProtocol will get modified. As AM-RM heartbeat is mandatory for all active AMs, adding
this will add burden on ApplicationMasterService.

* Allow ClientRMProtocol to accept AMToken too along with ClientDelegationToken. Thereby AM
can communicate with RM even on ClientRMProtocol. 
** Advantages: -No need to share/create ClientDelegationToken
** Disadvantages: -same as issuing ClientDelegationToken.
                
> Make it easier to access cluster topology information in an AM
> --------------------------------------------------------------
>
>                 Key: YARN-435
>                 URL: https://issues.apache.org/jira/browse/YARN-435
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Hitesh Shah
>            Assignee: Hitesh Shah
>
> ClientRMProtocol exposes a getClusterNodes api that provides a report on all nodes in
the cluster including their rack information. 
> However, this requires the AM to open and establish a separate connection to the RM in
addition to one for the AMRMProtocol. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message