From yarn-issues-return-155845-apmail-hadoop-yarn-issues-archive=hadoop.apache.org@hadoop.apache.org Fri Oct 19 21:40:03 2018 Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B622A18884 for ; Fri, 19 Oct 2018 21:40:03 +0000 (UTC) Received: (qmail 27827 invoked by uid 500); 19 Oct 2018 21:40:03 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 27773 invoked by uid 500); 19 Oct 2018 21:40:03 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 27762 invoked by uid 99); 19 Oct 2018 21:40:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 19 Oct 2018 21:40:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 2A874C1D25 for ; Fri, 19 Oct 2018 21:40:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.301 X-Spam-Level: X-Spam-Status: No, score=-110.301 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id QYmrZon0y4x4 for ; Fri, 19 Oct 2018 21:40:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 8BDB05F381 for ; Fri, 19 Oct 2018 21:40:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 87C1EE0F39 for ; Fri, 19 Oct 2018 21:40:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 18CC225296 for ; Fri, 19 Oct 2018 21:40:00 +0000 (UTC) Date: Fri, 19 Oct 2018 21:40:00 +0000 (UTC) From: "Eric Yang (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (YARN-8922) Fix test-container-executor MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-8922?page=3Dcom.atlassian.= jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D16657= 472#comment-16657472 ]=20 Eric Yang edited comment on YARN-8922 at 10/19/18 9:39 PM: ----------------------------------------------------------- The code base may have years of evolution that made some mode of operations= irreverent. There are two modes that container-executor operates on: In none secure mode yarn user =3D yarn.nodemanager.linux-container-executor.nonsecure-mode.loca= l-user (nobody) user =3D user who submitted the app. In secure mode yarn user =3D yarn (or the user who runs node manager) user =3D user who submitted the app When unit test running as normal user, the non-secure mode is exercised. A= dditional tests will be triggered for secure mode validations, if unit test= is running as root user. I think delete /tmp/test-container-executor after test-container-executor c= ompleted, is probably the good enough option to solve the unit tests proble= m. However, 755 must be enforced for yarn local dir to prevent security pr= oblem, and /tmp/test-container-executor seems like a prefix of yarn local d= ir. Without enforcing 755, it is a security hole as well. was (Author: eyang): The code base may have years of evolution that made some mode of operations= irreverent. There are two modes that container-executor operates on: In none secure mode yarn user =3D yarn.nodemanager.linux-container-executor.nonsecure-mode.loca= l-user (nobody) user =3D user who submitted the app. In secure mode yarn user =3D yarn (or the user who runs node manager) user =3D user who submitted the app When unit test running as normal user, the non-secure mode is exercised. A= dditional tests will be triggered for secure mode validations, if unit test= is running as root user. I think delete /tmp/test-container-executor after test-container-executor c= ompleted, is probably the better option to solve the unit tests problem. > Fix test-container-executor > --------------------------- > > Key: YARN-8922 > URL: https://issues.apache.org/jira/browse/YARN-8922 > Project: Hadoop YARN > Issue Type: Bug > Components: test > Affects Versions: 3.3.0 > Reporter: Robert Kanter > Assignee: Robert Kanter > Priority: Major > Attachments: YARN-8922.001.patch > > > YARN-8448 attempted to fix the {{test-container-executor}} C test=C2=A0to= be able to run as root. The test claims that it should be possible to run= as root; in fact, there are some tests that only run if you use root. =20 > One of the fixes was to change the permissions of the test's config dir t= o 0777 from 0755. The problem was that the directory was owned by root, bu= t then other users would need to write files/directories under it, which wo= uld fail with 0755. YARN-8448 fixed this by making it 0777. However, this= breaks running cetest because it expects the directory to be 0755, and it'= s run afterwards. > The proper fix for all this is to leave the directory at 0755, but to mak= e sure it's owned by the "nodemanager" user. Confusingly, in {{test-contai= ner-executor}}, that appears to be the {{username}} and not the {{yarn_user= name}} (i.e. {{username}} is the user running the NM while {{yarn_username}= } is just some user running a Yarn app). -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org