From yarn-issues-return-137010-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Thu Feb 1 22:33:09 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 02B75180652 for ; Thu, 1 Feb 2018 22:33:09 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id E63A3160C44; Thu, 1 Feb 2018 21:33:08 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 3A3F4160C35 for ; Thu, 1 Feb 2018 22:33:08 +0100 (CET) Received: (qmail 13591 invoked by uid 500); 1 Feb 2018 21:33:07 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 13580 invoked by uid 99); 1 Feb 2018 21:33:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Feb 2018 21:33:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id C76AE19A911 for ; Thu, 1 Feb 2018 21:33:06 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.311 X-Spam-Level: X-Spam-Status: No, score=-110.311 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 93uU8IY0dzL3 for ; Thu, 1 Feb 2018 21:33:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 02C715F3B9 for ; Thu, 1 Feb 2018 21:33:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 05105E0308 for ; Thu, 1 Feb 2018 21:33:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 3A31321E8A for ; Thu, 1 Feb 2018 21:33:00 +0000 (UTC) Date: Thu, 1 Feb 2018 21:33:00 +0000 (UTC) From: "Jim Brennan (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-7677) Docker image cannot set HADOOP_CONF_DIR MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349305#comment-16349305 ] Jim Brennan commented on YARN-7677: ----------------------------------- I have tested a version of the patch where I write out the whitelisted variables first, and it does work for my test cases. But looking at the launch_container.sh that is produced, the order of other variables is not the same as launch_container.sh from before my changes. Since the whitelisted variables are not added to the environment hash map, the order of traversal is different. I'm not comfortable with putting this change in as-is, because while the ordering differences I'm seeing are not a problem in my test cases, there is no guarantee that others would not run into problems due to this change. I discussed this with [~jlowe], and he pointed me at YARN-5714. I think we need to address that problem before putting this change in. > Docker image cannot set HADOOP_CONF_DIR > --------------------------------------- > > Key: YARN-7677 > URL: https://issues.apache.org/jira/browse/YARN-7677 > Project: Hadoop YARN > Issue Type: Bug > Affects Versions: 3.0.0 > Reporter: Eric Badger > Assignee: Jim Brennan > Priority: Major > Attachments: YARN-7677.001.patch, YARN-7677.002.patch > > > Currently, {{HADOOP_CONF_DIR}} is being put into the task environment whether it's set by the user or not. It completely bypasses the whitelist and so there is no way for a task to not have {{HADOOP_CONF_DIR}} set. This causes problems in the Docker use case where Docker containers will set up their own environment and have their own {{HADOOP_CONF_DIR}} preset in the image itself. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org