hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2928) Application Timeline Server (ATS) next gen: phase 1
Date Fri, 06 Feb 2015 22:59:37 GMT

    [ https://issues.apache.org/jira/browse/YARN-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310114#comment-14310114

Hitesh Shah commented on YARN-2928:

bq. In this use case, who is the user of the TEZ application? This may affect the data mode
and the parent-child relationship (cluster->user->flow->flow run->application).
When you say user, what does it really imply? User "a" can submit a hive query. A tez application
running as user "hive" can execute the query submitted by user "a" using a's delegation tokens.
With proxy users and potential use of delegation tokens, which "user" should be used? 

bq. "metadata" aims to store the same thing as "otherInfo", ... "primaryFilters" 
Seems like a good option. What form of search will be supported? In most cases, values will
unlikely be primitive types but deep nested structures. Will you support all forms of search
on all objects? 

bq. They sound to be interesting features, ..
My point related to events was not about a new interesting feature but to generally understand
what use case is meant to be solved by events and how should an application developer use

bq. We may probably run adhoc query to get the events of all applications of a workflow.
How is a workflow defined when an entity has 2 parents? Considering the tez-hive example,
do you agree that both a Hive Query and a Tez application are workflows and share some entities?


> Application Timeline Server (ATS) next gen: phase 1
> ---------------------------------------------------
>                 Key: YARN-2928
>                 URL: https://issues.apache.org/jira/browse/YARN-2928
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Priority: Critical
>         Attachments: ATSv2.rev1.pdf, ATSv2.rev2.pdf, Data model proposal v1.pdf
> We have the application timeline server implemented in yarn per YARN-1530 and YARN-321.
Although it is a great feature, we have recognized several critical issues and features that
need to be addressed.
> This JIRA proposes the design and implementation changes to address those. This is phase
1 of this effort.

This message was sent by Atlassian JIRA

View raw message