hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anupam Seth (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-3251) JobClient should have an option to only to talk to RM to get job status
Date Mon, 24 Oct 2011 15:41:33 GMT
JobClient should have an option to only to talk to RM to get job status

                 Key: MAPREDUCE-3251
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3251
             Project: Hadoop Map/Reduce
          Issue Type: Task
          Components: mrv2
    Affects Versions: 0.23.0
            Reporter: Anupam Seth

In 0.20.xxx, the JobClient while polling goes to JT to get the job status. With YARN, AM can
be launched on any port and the client will have to have ACL open to that port to talk to
AM and get the job status. When the client is within the same grid network access to AM is
not a problem. But some applications may have one installation per set of clusters and may
launch jobs even across such sets (on job trackers in another set of clusters). For that to
work only the JT port needs to be open currently. In case of YARN, all ports will have to
be opened up for things to work. That would be a security no-no.

There are two possible solutions:
  1) Make the job client only talk to RM (as an option) to get the job status. 
  2) Limit the range of ports AM can listen on.

Option 2) may not be favorable as there is no direct OS API to find a free port.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message