hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuan Gong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6338) MR AppMaster does not honor ephemeral port range
Date Mon, 14 Mar 2016 01:37:33 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15192600#comment-15192600
] 

Xuan Gong commented on MAPREDUCE-6338:
--------------------------------------

[~j.bridge]

bq. Could you please explain why ContainerManagementProtocolProxy opens proxy and what it
actually does ? 

AM needs to connect the NM to launch the containers.

{code}
ContainerLauncher #0 org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy:
Opening proxy : host:45353
ContainerLauncher #1 org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy:
Opening proxy : host:45358
{code}

The "host:45353" is your node address which is randomly provided.

bq. Is this proxy configurable ? Can I disable proxy ?

Do not think so.

> MR AppMaster does not honor ephemeral port range
> ------------------------------------------------
>
>                 Key: MAPREDUCE-6338
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6338
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mr-am, mrv2
>    Affects Versions: 2.6.0
>            Reporter: Frank Nguyen
>            Assignee: Frank Nguyen
>              Labels: BB2015-05-TBR
>         Attachments: MAPREDUCE-6338.002.patch, MAPREDUCE-6338.003.patch
>
>
> The MR AppMaster should only use port ranges defined in the yarn.app.mapreduce.am.job.client.port-range
property.  On initial startup of the MRAppMaster, it does use the port range defined in the
property.  However, it also opens up a listener on a random ephemeral port.  This is not the
Jetty listener.  It is another listener opened by the MRAppMaster via another thread and is
recognized by the RM.  Other nodes will try to communicate to it via that random port.  With
firewall settings on, the MR job will fail because the random port is not opened.  This problem
has caused others to have all OS ephemeral ports opened to have MR jobs run.
> This is related to MAPREDUCE-4079



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message