From yarn-issues-return-140848-apmail-hadoop-yarn-issues-archive=hadoop.apache.org@hadoop.apache.org Wed Mar 28 17:07:03 2018 Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 957A518ACD for ; Wed, 28 Mar 2018 17:07:03 +0000 (UTC) Received: (qmail 36474 invoked by uid 500); 28 Mar 2018 17:07:03 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 36423 invoked by uid 500); 28 Mar 2018 17:07:03 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 36412 invoked by uid 99); 28 Mar 2018 17:07:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Mar 2018 17:07:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id C3BCA1A167F for ; Wed, 28 Mar 2018 17:07:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -101.511 X-Spam-Level: X-Spam-Status: No, score=-101.511 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id pUXFc4H9dz_o for ; Wed, 28 Mar 2018 17:07:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 6CC845FAE6 for ; Wed, 28 Mar 2018 17:07:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id CC9AFE092B for ; Wed, 28 Mar 2018 17:07:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 3FF0B255E4 for ; Wed, 28 Mar 2018 17:07:00 +0000 (UTC) Date: Wed, 28 Mar 2018 17:07:00 +0000 (UTC) From: "Rohith Sharma K S (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417766#comment-16417766 ] Rohith Sharma K S commented on YARN-6936: ----------------------------------------- bq. Let's add the scope of the entities to each of the four methods OK, Does this modified sentence fine? {{Send the information of a number of conceptual entities in the scope of a YARN application to the timeline service v.2 collector.}}. Does all 4 API need to be modified with same way? For newer API, it should be out side scope of application also right? bq. Is it intended to extend updateAggregateStatus() so that sub application metrics are rolled up? I vaguely remember this we discussed in weekly call and decided to aggregate for both APIs. Because newer APIs write into both tables i.e entity and subapp table which. So aggregated metrics can also available in application scope as well. bq. The TimelineCollectorContext is bound to an application attempt. Adding a subApplicationWrite flag to TimelineCollectorContext may not be the most intuitive approach. How about we leave subApplicationWrite as a separate flag instead? I would inclined to send required information in record rather sending in parameter. This avoids compatibility in future. May be let's define newer record that contains context, ugi and subappwrite. cc :/ [~vrushalic] > [Atsv2] Retrospect storing entities into sub application table from client perspective > -------------------------------------------------------------------------------------- > > Key: YARN-6936 > URL: https://issues.apache.org/jira/browse/YARN-6936 > Project: Hadoop YARN > Issue Type: Sub-task > Reporter: Rohith Sharma K S > Assignee: Rohith Sharma K S > Priority: Major > Attachments: YARN-6936.000.patch, YARN-6936.001.patch > > > Currently YARN-6734 stores entities into sub application table only if doAs user and submitted users are different. This holds good for Tez kind of use cases. But AM runs as same as submitted user like MR also need to store entities in sub application table so that it could read entities without application id. > This would be a point of concern later stages when ATSv2 is deployed into production. This JIRA is to retrospect decision of storing entities into sub application table based on client side configuration driven rather than user driven. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: yarn-issues-help@hadoop.apache.org