From yarn-issues-return-141043-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Thu Mar 29 21:13:06 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 94963180671 for ; Thu, 29 Mar 2018 21:13:05 +0200 (CEST) Received: (qmail 10234 invoked by uid 500); 29 Mar 2018 19:13:04 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 10223 invoked by uid 99); 29 Mar 2018 19:13:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Mar 2018 19:13:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 15061C0A5F for ; Thu, 29 Mar 2018 19:13:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id xh4DTD7tqokR for ; Thu, 29 Mar 2018 19:13:03 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 6EDCE5FB93 for ; Thu, 29 Mar 2018 19:13:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 4A2D0E0E70 for ; Thu, 29 Mar 2018 19:13:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id D3DEF255F8 for ; Thu, 29 Mar 2018 19:13:00 +0000 (UTC) Date: Thu, 29 Mar 2018 19:13:00 +0000 (UTC) From: "Vrushali C (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419609#comment-16419609 ] Vrushali C commented on YARN-6936: ---------------------------------- Haibo, Rohith and I chatted about this in our weekly call. Summarizing some points: - we will add a new EntityType for SubApplications similar to the ones at https://github.com/apache/hadoop/tree/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice - This entity type will also include a member for application id. This information is already available in the FlowContext during the HBaseTimelineWriterImpl#write and can be stored as one of the columns under info. - The TimelineWriter#write API thus does not change nor does any other existing behavior or function. - Haibo brought up a good reference about YARN-3401 > [Atsv2] Retrospect storing entities into sub application table from client perspective > -------------------------------------------------------------------------------------- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Rohith Sharma K S > Assignee: Rohith Sharma K S > Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs user and submitted users are different. This holds good for Tez kind of use cases. But AM runs as same as submitted user like MR also need to store entities in sub application table so that it could read entities without application id. > This would be a point of concern later stages when ATSv2 is deployed into production. This JIRA is to retrospect decision of storing entities into sub application table based on client side configuration driven rather than user driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org