From yarn-issues-return-134675-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Thu Jan 11 20:17:09 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 8F522180656 for ; Thu, 11 Jan 2018 20:17:09 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 7F385160C23; Thu, 11 Jan 2018 19:17:09 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id C4ED4160C13 for ; Thu, 11 Jan 2018 20:17:08 +0100 (CET) Received: (qmail 12223 invoked by uid 500); 11 Jan 2018 19:17:07 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 12212 invoked by uid 99); 11 Jan 2018 19:17:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Jan 2018 19:17:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 4E83A1A095D for ; Thu, 11 Jan 2018 19:17:07 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.711 X-Spam-Level: X-Spam-Status: No, score=-100.711 tagged_above=-999 required=6.31 tests=[RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id Gg8SV0877GFJ for ; Thu, 11 Jan 2018 19:17:05 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 45D665F250 for ; Thu, 11 Jan 2018 19:17:04 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 42956E2578 for ; Thu, 11 Jan 2018 19:17:03 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id E7333255CF for ; Thu, 11 Jan 2018 19:17:00 +0000 (UTC) Date: Thu, 11 Jan 2018 19:17:00 +0000 (UTC) From: "Eric Badger (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-7516) Security check for untrusted docker image MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322785#comment-16322785 ] Eric Badger commented on YARN-7516: ----------------------------------- bq. The untrusted image will appear as privileged but all kernel privileges dropped and disconnected all devices to outside world. Therefore, it isn't really privileged, only appear as root in the docker container without real root privileges to outside world. I don't believe it disconnects all devices. According to the Docker documentation, running as privileged automatically gives it access to *all* devices. Removing --device in the case of a privileged container (as is done in this patch) is moot, since the point of --device is to give containers access to devices that they would normally only be able to access if they were privileged. https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities bq. If you are suggesting to disallow --privileged flag for untrusted image completely, then it will limit our ability to try out new images before verifying the untrusted image can be promoted to privileged registry. Does this help to explain the reason that we don't check registry is privileged in set_privileged flag? I still don't quite understand why you require a container to be privileged to try new images. > Security check for untrusted docker image > ----------------------------------------- > > Key: YARN-7516 > URL: https://issues.apache.org/jira/browse/YARN-7516 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Eric Yang > Assignee: Eric Yang > Attachments: YARN-7516.001.patch, YARN-7516.002.patch, YARN-7516.003.patch, YARN-7516.004.patch, YARN-7516.005.patch, YARN-7516.006.patch, YARN-7516.007.patch > > > Hadoop YARN Services can support using private docker registry image or docker image from docker hub. In current implementation, Hadoop security is enforced through username and group membership, and enforce uid:gid consistency in docker container and distributed file system. There is cloud use case for having ability to run untrusted docker image on the same cluster for testing. > The basic requirement for untrusted container is to ensure all kernel and root privileges are dropped, and there is no interaction with distributed file system to avoid contamination. We can probably enforce detection of untrusted docker image by checking the following: > # If docker image is from public docker hub repository, the container is automatically flagged as insecure, and disk volume mount are disabled automatically, and drop all kernel capabilities. > # If docker image is from private repository in docker hub, and there is a white list to allow the private repository, disk volume mount is allowed, kernel capabilities follows the allowed list. > # If docker image is from private trusted registry with image name like "private.registry.local:5000/centos", and white list allows this private trusted repository. Disk volume mount is allowed, kernel capabilities follows the allowed list. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org