From mapreduce-issues-return-92122-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Thu Jul 26 00:45:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id AE6A518062C for ; Thu, 26 Jul 2018 00:45:04 +0200 (CEST) Received: (qmail 1640 invoked by uid 500); 25 Jul 2018 22:45:03 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 1629 invoked by uid 99); 25 Jul 2018 22:45:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 Jul 2018 22:45:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 0D6B81A1249 for ; Wed, 25 Jul 2018 22:45:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id u4MSWa6qITrH for ; Wed, 25 Jul 2018 22:45:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 2C7CE5F3BC for ; Wed, 25 Jul 2018 22:45:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 5F1B5E0178 for ; Wed, 25 Jul 2018 22:45:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 22AB92775E for ; Wed, 25 Jul 2018 22:45:00 +0000 (UTC) Date: Wed, 25 Jul 2018 22:45:00 +0000 (UTC) From: "Robert Kanter (JIRA)" To: mapreduce-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (MAPREDUCE-4669) MRAM web UI does not work with HTTPS MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/MAPREDUCE-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556366#comment-16556366 ] Robert Kanter edited comment on MAPREDUCE-4669 at 7/25/18 10:44 PM: -------------------------------------------------------------------- See this comment in YARN-8448 and the design doc in YARN-6586 for more background on the patch. This patch (MAPREDUCE-4669.001.patch) contains the MR changes that rely on YARN-8448.001.patch. Some notes on the patch: - The {{yarn.app.mapreduce.am.webapp.https.enabled}} property controls if the MR AM should try to use the Yarn-provided keystore (when set to {{true}}); this will also cause it to provide an HTTPS tracking URL to the RM. It defaults to {{false}}. - The {{yarn.app.mapreduce.am.webapp.https.client.auth}} property controls if the MR AM should require client authentication (when set to {{true}}). It defaults to {{false}}. In this case, the MR AM is the server and the RM is the client, so this requires that the RM present its certificate to the AM when it connects to the AM - the AM can then verify this certificate with the Yarn-provided truststore. - It won't compile without the YARN-8448 patch. was (Author: rkanter): See [this comment|https://issues.apache.org/jira/browse/YARN-8448?focusedCommentId=16556364&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16556364] in YARN-8448 and the design doc in YARN-6586 for more background on the patch. This patch (MAPREDUCE-4669.001.patch) contains the MR changes that rely on YARN-8448.001.patch. Some notes on the patch: - The {{yarn.app.mapreduce.am.webapp.https.enabled}} property controls if the MR AM should try to use the Yarn-provided keystore (when set to {{true}}); this will also cause it to provide an HTTPS tracking URL to the RM. It defaults to {{false}}. - The {{yarn.app.mapreduce.am.webapp.https.client.auth}} property controls if the MR AM should require client authentication (when set to {{true}}). It defaults to {{false}}. In this case, the MR AM is the server and the RM is the client, so this requires that the RM present its certificate to the AM when it connects to the AM - the AM can then verify this certificate with the Yarn-provided truststore. > MRAM web UI does not work with HTTPS > ------------------------------------ > > Key: MAPREDUCE-4669 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4669 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am > Affects Versions: 2.0.3-alpha > Reporter: Alejandro Abdelnur > Assignee: Robert Kanter > Priority: Major > Attachments: MAPREDUCE-4669.001.patch > > > With Kerberos enable, the MRAM runs as the user that submitted the job, thus the MRAM process cannot read the cluster keystore files to get the certificates to start its HttpServer using HTTPS. > We need to decouple the keystore used by RM/NM/NN/DN (which are cluster provided) from the keystore used by AMs (which ought to be user provided). -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org