hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chun Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
Date Tue, 02 Jun 2015 03:22:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568448#comment-14568448

Chun Chen commented on YARN-3749:

Thanks for the review [~zxu] [~iwasakims].
Upload a new patch to address your comments.

bq. 1. It looks like setRpcAddressForRM and setConfForRM are only used by test code. Should
we create a new HA test utility file to include these functions?

Moved setRpcAddressForRM and setConfForRM to HATestUtil.java

bq. 2. Do we really need the following change at MiniYARNCluster#serviceInit conf.set(YarnConfiguration.RM_HA_ID,

This is indeed necessary, as [~iwasakims]'s comments, this is used to bypass the check in
`HAUtil#getRMHAId` used by NodeManager instance.

bq. 3. Is any particular reason to configure YarnConfiguration.RM_HA_ID as RM2_NODE_ID instead
of RM1_NODE_ID in ProtocolHATestBase?

Not really, changed it to RM1_NODE_ID.

bq. I think there should be a comment explain that it is a dummy for unit test at least.

Added a comment in `MiniYARNCluster#serviceInit`

Also the newly uploaded patch YARN-3749.4.patch only make a copy of the configuration in initResourceManager
when there are multiple RMs. If there is only one RM, many test case in yarn-client depends
on the random ports after RM starts.

> We should make a copy of configuration when init MiniYARNCluster with multiple RMs
> ----------------------------------------------------------------------------------
>                 Key: YARN-3749
>                 URL: https://issues.apache.org/jira/browse/YARN-3749
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Chun Chen
>            Assignee: Chun Chen
>         Attachments: YARN-3749.2.patch, YARN-3749.3.patch, YARN-3749.4.patch, YARN-3749.patch
> When I was trying to write a test case for YARN-2674, I found DS client trying to connect
to both rm1 and rm2 with the same address when RM failover. But I initially
set yarn.resourcemanager.address.rm1=, yarn.resourcemanager.address.rm2=
 After digging, I found it is in ClientRMService where the value of yarn.resourcemanager.address.rm2
changed to See the following code in ClientRMService:
> {code}
> clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
>                                                YarnConfiguration.RM_ADDRESS,
>                                                YarnConfiguration.DEFAULT_RM_ADDRESS,
>                                                server.getListenerAddress());
> {code}
> Since we use the same instance of configuration in rm1 and rm2 and init both RM before
we start both RM, we will change yarn.resourcemanager.ha.id to rm2 during init of rm2 and
yarn.resourcemanager.ha.id will become rm2 during starting of rm1.
> So I think it is safe to make a copy of configuration when init both of the rm.

This message was sent by Atlassian JIRA

View raw message