hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6936) [Atsv2] Retrospect storing entities into sub application table from client perspective
Date Wed, 28 Mar 2018 14:36:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417430#comment-16417430

Haibo Chen commented on YARN-6936:

Thanks [~rohithsharma] for the new patch. I have a few comments/questions

1) The javadoc of the two new method in TimelineV2Client, are the same as that of the existing
two putEntitites() and putEntitiesAsync() methods. Let's add the scope of the entities to
each of the four methods. That is, for putEntities() and putEntitiesAsync(), say 'conceptual
entities in the scope of a YARN application rather than 'conceptual entities'. Similarly for
putSubAppEntities() and putSubAppEntitiesAsync()

2) In TimelineCollector, we'd call updateAggregateStatus() for each entity, regardless of
whether subApp entity or not. IIRC, updateAggregateStatus() is for application-level metrics
aggregation. Is it intended to extend updateAggregateStatus() so that sub application metrics
are rolled up?

3) The TimelineCollectorContext is bound to an application attempt. Adding a subApplicationWrite
flag to TimelineCollectorContext may not be the most intuitive approach. How about we leave
subApplicationWrite as a separate flag instead?

> [Atsv2] Retrospect storing entities into sub application table from client perspective
> --------------------------------------------------------------------------------------
>                 Key: YARN-6936
>                 URL: https://issues.apache.org/jira/browse/YARN-6936
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Major
>         Attachments: YARN-6936.000.patch, YARN-6936.001.patch
> Currently YARN-6734 stores entities into sub application table only if doAs user and
submitted users are different. This holds good for Tez kind of use cases. But AM runs as same
as submitted user like MR also need to store entities in sub application table so that it
could read entities without application id. 
> This would be a point of concern later stages when ATSv2 is deployed into production.
This JIRA is to retrospect decision of storing entities into sub application table based on
client side configuration driven rather than user driven. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message