hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
Date Wed, 28 Mar 2018 17:07:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417766#comment-16417766
] 

Rohith Sharma K S commented on YARN-6936:
-----------------------------------------

bq. Let's add the scope of the entities to each of the four methods
OK, Does this modified sentence fine? {{Send the information of a number of conceptual entities
in the scope of a YARN application to the timeline service v.2 collector.}}. Does all 4 API
need to be modified with same way? For newer API, it should be out side scope of application
also right?

bq.  Is it intended to extend updateAggregateStatus() so that sub application metrics are
rolled up?
I vaguely remember this we discussed in weekly call and decided to aggregate for both APIs.
Because newer APIs write into both tables i.e entity and subapp table which. So aggregated
metrics can also available in application scope as well. 

bq. The TimelineCollectorContext is bound to an application attempt. Adding a subApplicationWrite
flag to TimelineCollectorContext may not be the most intuitive approach. How about we leave
subApplicationWrite as a separate flag instead?
I would inclined to send required information in record rather sending in parameter. This
avoids compatibility in future. May be let's define newer record that contains context, ugi
and subappwrite.  cc :/ [~vrushalic]



> [Atsv2] Retrospect storing entities into sub application table from client perspective
> --------------------------------------------------------------------------------------
>
>                 Key: YARN-6936
>                 URL: https://issues.apache.org/jira/browse/YARN-6936
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Major
>         Attachments: YARN-6936.000.patch, YARN-6936.001.patch
>
>
> Currently YARN-6734 stores entities into sub application table only if doAs user and
submitted users are different. This holds good for Tez kind of use cases. But AM runs as same
as submitted user like MR also need to store entities in sub application table so that it
could read entities without application id. 
> This would be a point of concern later stages when ATSv2 is deployed into production.
This JIRA is to retrospect decision of storing entities into sub application table based on
client side configuration driven rather than user driven. 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message