hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Anand (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-7592) yarn.federation.failover.enabled missing in yarn-default.xml
Date Mon, 24 Sep 2018 15:47:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625870#comment-16625870
] 

Rahul Anand edited comment on YARN-7592 at 9/24/18 3:46 PM:
------------------------------------------------------------

Thanks [~bibinchundatt] and [~subru].

Removing *yarn.federation.enabled* from yarn-site.xml can solve this issue but would definitely
create a confusion. So, instead of changing/removing a meaningful federation flag or updating
doc, an alternative solution can be creation of a {{FederationCustomClientRMProxy}} which
can override the {{ClientRMProxy#createRMProxy}} in {{AMRMClientUtils}} to always select
*proxy provider* as {{FederationRMFailoverProxyProvider}} for federation.
{code:java}
public static <T> T createRMProxy(final Configuration configuration,
      final Class<T> protocol, UserGroupInformation user,
      final Token<? extends TokenIdentifier> token) throws IOException {
     ...
          return FederationCustomClientRMProxy.createRMProxy(configuration, protocol);
        }
     ...
    }
  }
{code}
After this, we can remove the {{isFederationEnabled}} check from the {{RMProxy.java}} as before. 
{code:java}
protected static <T> T createRMProxy(final Configuration configuration,
      final Class<T> protocol, RMProxy<T> instance) throws IOException {
    ...
    RetryPolicy retryPolicy = createRetryPolicy(conf,
        (HAUtil.isHAEnabled(conf)));
    ...
  }
{code}
{code:java}
  protected static <T> T createRMProxy(final Configuration configuration,
      final Class<T> protocol, RMProxy<T> instance, final long retryTime,
      final long retryInterval) throws IOException {
    ...
    RetryPolicy retryPolicy = createRetryPolicy(conf, retryTime, retryInterval,
        HAUtil.isHAEnabled(conf));
    ...
  }
{code}
With this change we don't need to separately specify the *proxy provider* for HA and non-HA
scenarios in case of federation while other non federation settings will continue as it is.


was (Author: rahulanand90):
Thanks [~bibinchundatt] and [~subru]. 

Removing *yarn.federation.enabled* from yarn-site.xml can solve this issue but would definitely
create a confusion. So, instead of changing/removing a meaningful federation flag or updating
doc, an alternative solution can be creation of a {{FederationCustomClientRMProxy}} which
can override the {{ClientRMProxy#createRMProxy}} in {{AMRMClientUtils}} to always select
*proxy provider* as {{FederationRMFailoverProxyProvider}} for federation.
{code:java}
public static <T> T createRMProxy(final Configuration configuration,
      final Class<T> protocol, UserGroupInformation user,
      final Token<? extends TokenIdentifier> token) throws IOException {
     ...
          return FederationCustomClientRMProxy.createRMProxy(configuration, protocol);
        }
     ...
    }
  }
{code}
After this, we can remove the {{isFederationEnabled}} check from the {{RMProxy.java}} as before. 
{code:java}
protected static <T> T createRMProxy(final Configuration configuration,
      final Class<T> protocol, RMProxy<T> instance) throws IOException {
    ...
    RetryPolicy retryPolicy = createRetryPolicy(conf,
        (HAUtil.isHAEnabled(conf)));
    ...
  }
{code}
{code:java}
  protected static <T> T createRMProxy(final Configuration configuration,
      final Class<T> protocol, RMProxy<T> instance, final long retryTime,
      final long retryInterval) throws IOException {
    ...
    RetryPolicy retryPolicy = createRetryPolicy(conf, retryTime, retryInterval,
        HAUtil.isHAEnabled(conf));
    ...
  }
{code}
With this change, we don't need to seperately  specify the *proxy provider* for HA and non-HA
scenarios.

> yarn.federation.failover.enabled missing in yarn-default.xml
> ------------------------------------------------------------
>
>                 Key: YARN-7592
>                 URL: https://issues.apache.org/jira/browse/YARN-7592
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: federation
>    Affects Versions: 3.0.0-beta1
>            Reporter: Gera Shegalov
>            Priority: Major
>         Attachments: IssueReproduce.patch
>
>
> yarn.federation.failover.enabled should be documented in yarn-default.xml. I am also
not sure why it should be true by default and force the HA retry policy in {{RMProxy#createRMProxy}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message