hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3528) Tests with 12345 as hard-coded port break jenkins
Date Sat, 25 Jul 2015 17:44:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14641736#comment-14641736
] 

Varun Saxena commented on YARN-3528:
------------------------------------

[~brahmareddy],

Few comments :

# Amongst the test failures which have come in QA report above, except one test each in TestDeletionService
and TestResourceLocalizationService, all are related to your code change. 
# You need to make a call to {{ServerSocketUtil#getPort}} only in places where the port is
used for binding to a socket. You have used it elsewhere as well.
# In {{TestNodeManagerReboot#createNMConfig}}, below changes would lead to bind exception.
Because at the time of setting config, call to ServerSocketUtil#getPort will return 49152(if
free) and both NM_ADDRESS and NM_LOCALIZER_ADDRESS will be set using same port. The start
port should be different at the time of second call to getPort.
{code}
-      conf.set(YarnConfiguration.NM_ADDRESS, "127.0.0.1:12345");
-      conf.set(YarnConfiguration.NM_LOCALIZER_ADDRESS, "127.0.0.1:12346");
+      conf.set(YarnConfiguration.NM_ADDRESS,
+          "127.0.0.1:" + ServerSocketUtil.getPort(49152, 10));
+      conf.set(YarnConfiguration.NM_LOCALIZER_ADDRESS, "127.0.0.1:"
+          + ServerSocketUtil.getPort(49152, 10));
{code}
# Same problem as above exists in {{TestNodeManagerResync#createNMConfig}} which will lead
to test failures due to BindException.
# In {{TestNMContainerTokenSecretManager}}, {{TestRMAppLogAggregationStatus}} and {{TestNMTokenSecretManagerInNM}},
you do not need to get port from ServerSocketUtil to set NodeID
# In {{TestNMWebServer#testNMWebApp}}, you do not need to call ServerSocketUtil#getPort to
create the token. Token is not used for socket binding.
# In {{TestRMApplicationHistoryWriter}} and {{TestAMRMRPCResponseId}}, call to MockRM#registerNode
does not need a unique port to bind to. So again we do not need to get port from ServerSocketUtil
# Same applies for {{TestAMRestart}} wherever MockNM constructor is invoked.
# In {{TestAMRestart}}, you need to have different ports wherever more than one MockNM object
is created. This is leading to test failure in QA report above.
# Nit: Formatting of below piece of code in {{TestNMWebServer}} is not correct.
{code}
       Token containerToken =
-          BuilderUtils.newContainerToken(containerId, "127.0.0.1", 1234, user,
+ BuilderUtils.newContainerToken(containerId,
+          "127.0.0.1", ServerSocketUtil.getPort(49152, 10), user,
{code}


> Tests with 12345 as hard-coded port break jenkins
> -------------------------------------------------
>
>                 Key: YARN-3528
>                 URL: https://issues.apache.org/jira/browse/YARN-3528
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>         Environment: ASF Jenkins
>            Reporter: Steve Loughran
>            Assignee: Brahma Reddy Battula
>            Priority: Blocker
>              Labels: test
>         Attachments: YARN-3528-002.patch, YARN-3528-003.patch, YARN-3528-004.patch, YARN-3528.patch
>
>
> A lot of the YARN tests have hard-coded the port 12345 for their services to come up
on.
> This makes it impossible to have scheduled or precommit tests to run consistently on
the ASF jenkins hosts. Instead the tests fail regularly and appear to get ignored completely.
> A quick grep of "12345" shows up many places in the test suite where this practise has
developed.
> * All {{BaseContainerManagerTest}} subclasses
> * {{TestNodeManagerShutdown}}
> * {{TestContainerManager}}
> + others
> This needs to be addressed through portscanning and dynamic port allocation. Please can
someone do this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message