hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Kanter (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3528) Tests with 12345 as hard-coded port break jenkins
Date Fri, 29 May 2015 02:18:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14564091#comment-14564091
] 

Robert Kanter commented on YARN-3528:
-------------------------------------

I don't think we can simply put 0 everywhere.  In some places it looks like we set a config
(e.g. NM address) and then some other code actually starts using the port based on the config.
 In that case, I think we can do something like this to choose a port:
{code:java}

    public int getPort() throws RuntimeException {
      Random rand = new Random();
      int port = -1;
      int tries = 0;
      while(port == -1) {
        ServerSocket s = null;
        try {
          int tryPort = 49152 + rand.nextInt(65535 - 49152);
          System.out.println("Using port " + tryPort);
          s = new ServerSocket(tryPort);
          port = tryPort;
        } catch (IOException e) {
          tries++;
          if (tries >= 10) {
            System.out.println("Port is already in use; giving up");
            throw new RuntimeException(e);
          } else {
            System.out.println("Port is already in use; trying again");
          }
        } finally {
          IOUtils.closeQuietly(s);
        }
      }
      return port;
    }
{code}
It's possible for something to steal the port between the time this method finds an open one
and the time that we actually start using it, but it's not likely.

Thoughts?

> Tests with 12345 as hard-coded port break jenkins
> -------------------------------------------------
>
>                 Key: YARN-3528
>                 URL: https://issues.apache.org/jira/browse/YARN-3528
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>         Environment: ASF Jenkins
>            Reporter: Steve Loughran
>            Assignee: Brahma Reddy Battula
>            Priority: Blocker
>              Labels: test
>
> A lot of the YARN tests have hard-coded the port 12345 for their services to come up
on.
> This makes it impossible to have scheduled or precommit tests to run consistently on
the ASF jenkins hosts. Instead the tests fail regularly and appear to get ignored completely.
> A quick grep of "12345" shows up many places in the test suite where this practise has
developed.
> * All {{BaseContainerManagerTest}} subclasses
> * {{TestNodeManagerShutdown}}
> * {{TestContainerManager}}
> + others
> This needs to be addressed through portscanning and dynamic port allocation. Please can
someone do this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message