From yarn-issues-return-143893-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Tue May 1 23:16:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4CE19180675 for ; Tue, 1 May 2018 23:16:05 +0200 (CEST) Received: (qmail 41184 invoked by uid 500); 1 May 2018 21:16:03 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 41173 invoked by uid 99); 1 May 2018 21:16:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 May 2018 21:16:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 31D5E1804C8 for ; Tue, 1 May 2018 21:16:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.311 X-Spam-Level: X-Spam-Status: No, score=-110.311 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id dRbUcpMm9eHh for ; Tue, 1 May 2018 21:16:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 9089B5FBA7 for ; Tue, 1 May 2018 21:16:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 1A8D6E12AE for ; Tue, 1 May 2018 21:16:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 78F13212A5 for ; Tue, 1 May 2018 21:16:00 +0000 (UTC) Date: Tue, 1 May 2018 21:16:00 +0000 (UTC) From: "Gour Saha (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-7799) YARN Service dependency follow up work MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-7799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460167#comment-16460167 ] Gour Saha commented on YARN-7799: --------------------------------- Ok, I think it makes sense. We can do the system service later. So just to make sure I understand the latest patch, only if a user with HDFS admin ACLs (like hdfs user) submits a service launch is when the dependency tarball will get uploaded, right? > YARN Service dependency follow up work > -------------------------------------- > > Key: YARN-7799 > URL: https://issues.apache.org/jira/browse/YARN-7799 > Project: Hadoop YARN > Issue Type: Bug > Components: client, resourcemanager > Reporter: Gour Saha > Assignee: Billie Rinaldi > Priority: Critical > Attachments: YARN-7799.1.patch, YARN-7799.2.patch, YARN-7799.3.patch, YARN-7799.4.patch, YARN-7799.5.patch > > > As per [~jianhe] these are some followup items that make sense to do after YARN-7766. Quoting Jian's comment below - > Currently, if user doesn't supply location when run yarn app -enableFastLaunch, the jars will be put under this location > {code} > hdfs:///yarn-services//service-dep.tar.gz > {code} > Since API server is embedded in RM, should RM look for this location too if "yarn.service.framework.path" is not specified ? > And if "yarn.service.framework.path" is not specified and still the file doesn't exist at above default location, I think RM can try to upload the jars to above default location instead, currently RM is uploading the jars to the location defined by below code. This folder is per app and also inconsistent with CLI location. > {code} > protected Path addJarResource(String serviceName, > Map localResources) > throws IOException, SliderException { > Path libPath = fs.buildClusterDirPath(serviceName); > {code} > By doing this, the next time a submission request comes, RM doesn't need to upload the jars again. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org