hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3251) Network ACLs can prevent some clients to talk to MR ApplicationMaster
Date Tue, 20 Dec 2011 22:49:31 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinod Kumar Vavilapalli updated MAPREDUCE-3251:
-----------------------------------------------

    Status: Open  (was: Patch Available)

This looks better.

 - Atleast for now, the configuration is a MapReduce-only flag and definitely not related
to resourceManager. Let's rename it as {{mapreduce.job.am-access-disabled}} and move it to
{{MRJobConfig}}.
 - Not sure why logApplicationReportInfo() is needed. Let's drop this unless you did it explicitly
for some reason.
 - Correct the log statement "Network ACL closed to AM for job " + jobId + ". Redirecting
to job history server." We aren't redirecting to the history server.
 - Can you add a new test in {{TestClientServiceDelegate}}? None of the tests which run in
the access-disabled mode do not explicitly test the current code. We need something like this:
   -- Client goes to RM, gets running state
   -- Tries to create a proxy, but doesn't reach the AM even though AM is alive, while the
job is running
   -- Keeps doing the above till the job completes
   -- on job-completion, the client goes to the history-server.
                
> Network ACLs can prevent some clients to talk to MR ApplicationMaster
> ---------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3251
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3251
>             Project: Hadoop Map/Reduce
>          Issue Type: Task
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Anupam Seth
>            Assignee: Anupam Seth
>            Priority: Critical
>             Fix For: 0.23.1
>
>         Attachments: MAPREDUCE-3251-branch_0_23.patch, MAPREDUCE-3251-branch_0_23.patch,
MAPREDUCE-3251-branch_0_23.patch, MAPREDUCE-3251-branch_0_23.patch, MAPREDUCE-3251-branch_0_23_incremental_fix.patch,
MAPREDUCE-3251_branch-0_23_preliminary.txt
>
>
> In 0.20.xxx, the JobClient while polling goes to JT to get the job status. With YARN,
AM can be launched on any port and the client will have to have ACL open to that port to talk
to AM and get the job status. When the client is within the same grid network access to AM
is not a problem. But some applications may have one installation per set of clusters and
may launch jobs even across such sets (on job trackers in another set of clusters). For that
to work only the JT port needs to be open currently. In case of YARN, all ports will have
to be opened up for things to work. That would be a security no-no.
> There are two possible solutions:
>   1) Make the job client only talk to RM (as an option) to get the job status. 
>   2) Limit the range of ports AM can listen on.
> Option 2) may not be favorable as there is no direct OS API to find a free port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message