From yarn-issues-return-140822-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Wed Mar 28 16:36:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 906A0180652 for ; Wed, 28 Mar 2018 16:36:04 +0200 (CEST) Received: (qmail 42683 invoked by uid 500); 28 Mar 2018 14:36:03 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 42672 invoked by uid 99); 28 Mar 2018 14:36:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Mar 2018 14:36:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 15EFAC0047 for ; Wed, 28 Mar 2018 14:36:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.511 X-Spam-Level: X-Spam-Status: No, score=-109.511 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id oAs1x7V-H5Ww for ; Wed, 28 Mar 2018 14:36:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 361825F36B for ; Wed, 28 Mar 2018 14:36:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6D4DDE0842 for ; Wed, 28 Mar 2018 14:36:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 2AE88255DF for ; Wed, 28 Mar 2018 14:36:00 +0000 (UTC) Date: Wed, 28 Mar 2018 14:36:00 +0000 (UTC) From: "Haibo Chen (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417430#comment-16417430 ] Haibo Chen commented on YARN-6936: ---------------------------------- Thanks [~rohithsharma] for the new patch. I have a few comments/questions 1) The javadoc of the two new method in TimelineV2Client, are the same as that of the existing two putEntitites() and putEntitiesAsync() methods. Let's add the scope of the entities to each of the four methods. That is, for putEntities() and putEntitiesAsync(), say 'conceptual entities in the scope of a YARN application rather than 'conceptual entities'. Similarly for putSubAppEntities() and putSubAppEntitiesAsync() 2) In TimelineCollector, we'd call updateAggregateStatus() for each entity, regardless of whether subApp entity or not. IIRC, updateAggregateStatus() is for application-level metrics aggregation. Is it intended to extend updateAggregateStatus() so that sub application metrics are rolled up? 3) The TimelineCollectorContext is bound to an application attempt. Adding a subApplicationWrite flag to TimelineCollectorContext may not be the most intuitive approach. How about we leave subApplicationWrite as a separate flag instead? > [Atsv2] Retrospect storing entities into sub application table from client perspective > -------------------------------------------------------------------------------------- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Rohith Sharma K S > Assignee: Rohith Sharma K S > Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs user and submitted users are different. This holds good for Tez kind of use cases. But AM runs as same as submitted user like MR also need to store entities in sub application table so that it could read entities without application id. > This would be a point of concern later stages when ATSv2 is deployed into production. This JIRA is to retrospect decision of storing entities into sub application table based on client side configuration driven rather than user driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org