Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 24 Jul 2017 23:29:00 +0000 (UTC)
From: "Vrushali C (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13081963.1498179843000.2277.1500938940181@Atlassian.JIRA>
In-Reply-To: <JIRA.13081963.1498179843000@Atlassian.JIRA>
References: <JIRA.13081963.1498179843000@Atlassian.JIRA> <JIRA.13081963.1498179843639@jira-lw-us.apache.org>
Subject: [jira] [Comment Edited] (YARN-6733) Add table for storing
 sub-application entities
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 24 Jul 2017 23:29:07 -0000


    [ https://issues.apache.org/jira/browse/YARN-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099207#comment-16099207 ] 

Vrushali C edited comment on YARN-6733 at 7/24/17 11:28 PM:
------------------------------------------------------------

So we thought that it will be good to keep the column name so that sub apps can store this information. For regular applications, the flow version can be used to determine whether optimizations are to be done. The flow version indicates if the flow has changed, that is, say if the pig script changes, its flow version will change. So then, for example, reducer estimation calculations can be done differently. This applies to the application entities. We discussed that it will be good to keep the same information for sub-apps in case they want to use this information in a similar fashion. As such, this column currently only exists in code, it's not taking up any disk space/hbase space etc if no one writes to it. But having it gives the framework developers a chance to use it if they want. 


was (Author: vrushalic):
So we thought that it will be good to keep the column name so that sub apps can store this information. For regular applications, the flow version can be used to determine whether optimizations are to be done. The flow version indicates if the flow has changed, that is, say if the pig script changes, it's flow version will change. So then, for example, reducer estimation calculations can be done differently. This applies to the application entities. We discussed that it will be good to keep the same information for sub-apps in case they want to use this information in a similar fashion. As such, this column currently only exists in code, it's not taking up any disk space/hbase space etc if no one writes to it. But having it given the framework developers a chance to use it if they want. 

> Add table for storing sub-application entities
> ----------------------------------------------
>
>                 Key: YARN-6733
>                 URL: https://issues.apache.org/jira/browse/YARN-6733
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Vrushali C
>            Assignee: Vrushali C
>         Attachments: IMG_7040.JPG, YARN-6733-YARN-5355.001.patch, YARN-6733-YARN-5355.002.patch, YARN-6733-YARN-5355.003.patch, YARN-6733-YARN-5355.004.patch, YARN-6733-YARN-5355.005.patch, YARN-6733-YARN-5355.006.patch, YARN-6733-YARN-5355.007.patch, YARN-6733-YARN-5355.008.patch
>
>
> After a discussion with Tez folks, we have been thinking over introducing a table to store  sub-application information.
> For example, if a Tez session runs for a certain period as User X and runs a few AMs. These AMs accept DAGs from other users. Tez will execute these dags with a doAs user. ATSv2 should store this information in a new table perhaps called as "sub_application" table. 
> This jira tracks the code changes needed for  table schema creation.
> I will file other jiras for writing to that table, updating the user name fields to include sub-application user etc.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org