hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
Date Thu, 29 Mar 2018 19:13:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419609#comment-16419609
] 

Vrushali C commented on YARN-6936:
----------------------------------

Haibo, Rohith and I chatted about this in our weekly call. Summarizing some points:

- we will add a new EntityType for SubApplications similar to the ones at https://github.com/apache/hadoop/tree/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice

- This entity type will also include a member for application id. This information is already
available in the FlowContext during the HBaseTimelineWriterImpl#write and can be stored as
one of the columns under info. 

- The TimelineWriter#write API thus does not change nor does any other existing behavior or
function. 

- Haibo brought up a good reference about YARN-3401




 

> [Atsv2] Retrospect storing entities into sub application table from client perspective
> --------------------------------------------------------------------------------------
>
>                 Key: YARN-6936
>                 URL: https://issues.apache.org/jira/browse/YARN-6936
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Major
>         Attachments: YARN-6936.000.patch, YARN-6936.001.patch
>
>
> Currently YARN-6734 stores entities into sub application table only if doAs user and
submitted users are different. This holds good for Tez kind of use cases. But AM runs as same
as submitted user like MR also need to store entities in sub application table so that it
could read entities without application id. 
> This would be a point of concern later stages when ATSv2 is deployed into production.
This JIRA is to retrospect decision of storing entities into sub application table based on
client side configuration driven rather than user driven. 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message