Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Tue, 15 Nov 2011 15:12:51 +0000 (UTC)
From: "Vinod Kumar Vavilapalli (Commented) (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: 
 <1221408784.31144.1321369971970.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <324233916.8951.1319470893154.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (MAPREDUCE-3251) Network ACLs can prevent some
 clients to talk to MR ApplicationMaster
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13150544#comment-13150544 ] 

Vinod Kumar Vavilapalli commented on MAPREDUCE-3251:
----------------------------------------------------

Back and forth on this one.

I went back and looked at an oozie console in action. Checked both running and completed jobs. Fortunately(and rightly), oozie restricts itself to the workflow level and doesn't peek into the mapreduce bits like progress, counters etc. It instead just points to the job's web-page. So, I think we are good if we just have a mode(a configuration that oozie can explicitly set) to circumvent communication with the not-reachable-due-to-ACLs MR ApplicationMasters.

Sure, there can be use-cases beyond oozie that may hit this issue. They can probably make do with the web-proxy that we have in RM.
                
> Network ACLs can prevent some clients to talk to MR ApplicationMaster
> ---------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3251
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3251
>             Project: Hadoop Map/Reduce
>          Issue Type: Task
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Anupam Seth
>            Assignee: Anupam Seth
>            Priority: Critical
>             Fix For: 0.23.1
>
>
> In 0.20.xxx, the JobClient while polling goes to JT to get the job status. With YARN, AM can be launched on any port and the client will have to have ACL open to that port to talk to AM and get the job status. When the client is within the same grid network access to AM is not a problem. But some applications may have one installation per set of clusters and may launch jobs even across such sets (on job trackers in another set of clusters). For that to work only the JT port needs to be open currently. In case of YARN, all ports will have to be opened up for things to work. That would be a security no-no.
> There are two possible solutions:
>   1) Make the job client only talk to RM (as an option) to get the job status. 
>   2) Limit the range of ports AM can listen on.
> Option 2) may not be favorable as there is no direct OS API to find a free port.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira