hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Kanter (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-6586) YARN to facilitate HTTPS in AM web server
Date Thu, 14 Jun 2018 23:13:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16513114#comment-16513114
] 

Robert Kanter edited comment on YARN-6586 at 6/14/18 11:12 PM:
---------------------------------------------------------------

I was initially thinking we would want the expiration dates on the child certificates because
they should eventually expire for safety, but given that the CN is the AppID and not a host,
you'd get browser warnings if someone were to try to reuse the cert elsewhere anyway.  So
now I'm thinking that there's no real advantage to worrying about the child cert expiration
and we could set it to +10 years or something.  That would also remove the complication about
long-running AMs.  Alternatively, instead of setting a long expiration, we could also set
a really short one and have the RM ignore the expiration on child certs - that might be safer
from an "outside-the-RM" perspective.


was (Author: rkanter):
I was initially thinking we would want the expiration dates on the child certificates because
they should eventually expire for safety, but given that the CN is the AppID and not a host,
you'd get browser warnings if someone were to try to reuse the cert elsewhere anyway.  So
now I'm thinking that there's no real advantage to worrying about the child cert expiration
and we could set it to +10 years or something.  That would also remove the complication about
long-running AMs.  Alternatively, instead of setting a long expiration, we could also set
a really short one and have the RM ignore expired child certs - that might be safer from an
"outside-the-RM" perspective.

> YARN to facilitate HTTPS in AM web server
> -----------------------------------------
>
>                 Key: YARN-6586
>                 URL: https://issues.apache.org/jira/browse/YARN-6586
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: yarn
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Haibo Chen
>            Assignee: Robert Kanter
>            Priority: Major
>         Attachments: Design Document v1.pdf, YARN-6586.poc.patch
>
>
> MR AM today does not support HTTPS in its web server, so the traffic between RMWebproxy
and MR AM is in clear text.
> MR cannot easily achieve this mainly because MR AMs are untrusted by YARN. A potential
solution purely within MR, similar to what Spark has implemented, is to allow users, when
they enable HTTPS in MR job, to provide their own keystore file, and then the file is uploaded
to distributed cache and localized for MR AM container. The configuration users need to do
is complex.
> More importantly, in typical deployments, however, web browsers go through RMWebProxy
to indirectly access MR AM web server. In order to support MR AM HTTPs, RMWebProxy therefore
needs to trust the user-provided keystore, which is problematic.  
> Alternatively, we can add an endpoint in NM web server that acts as a proxy between AM
web server and RMWebProxy. RMWebproxy, when configured to do so, will send requests in HTTPS
to the NM on which the AM is running, and the NM then can communicate with the local AM web
server in HTTP.   This adds one hop between RMWebproxy and AM, but both MR and Spark can use
such solution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message