hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4224) Change the ATSv2 reader side REST interface to conform to current REST APIs' in YARN
Date Wed, 23 Dec 2015 00:23:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15068950#comment-15068950
] 

Li Lu commented on YARN-4224:
-----------------------------

Thanks Varun. Some of my comments:
bq. You mean just have one endpoint and based on delimiters in UID, decide whether to fetch
single entity or multiple ?
It's about locating resources in the system. /flows will be a query endpoint and we can send
query parameters there, but /flows/\{uid\} will locate one single flow. What I'm confusing
right now is, why do we need to have both plural and singular forms /run/\{uid\} and /runs/\{uid\}?
Will they locate to the same run, given the same UID? 

bq. Because, the query before query for entities, in which we return UID, will be query for
apps. This query is from application table where we do not have entity related information.
So I cannot send all the possible entity types for an entity or a list of entities in the
response. Hence when we query list of entities, it is within the scope of app UID and entity
type. Hence entity type has to be specified.
bq. Yes, if we try to query all entity types and related entities, this would require scanning
quite a bit of the entity table which can grow quite big. And from UI, I envisage only queries
for APP_ATTEMPT and CONTAINER so we would know the entity type.

So the UID in /entities/\{entitytype\}/\{uid\}/ is actually app UID? This make the whole endpoint
looks really weird... I thought it's an entity UID to locate to one timeline entity. However,
I think you raised a very useful use case to query a certain type of entity for one application.
Maybe we'd like to change the format of this endpoint to address this case? I don't really
feel like the current form of the endpoint...

bq. Moreover, runs endpoint will do it for you i.e. fetch all flowruns for a flow.
OK this works for now. In future, if flows are associated with flow level aggregation data,
we will need endpoints to retrieve flow level data. We can skip this step for our first milestone
though. 

bq. There is. The \/entity\/{uid}\/ endpoint. I hope this is what your question was.
So to find one entity with cluster, user, flow, flowrun, appid and entity id, we do not have
the hierarchical endpoint, but can only get an entity through the UID interface? Do we need
the hierarchical interface for CLIs? 

We can certainly discuss more of these in our meeting tomorrow. 


> Change the ATSv2 reader side REST interface to conform to current REST APIs' in YARN
> ------------------------------------------------------------------------------------
>
>                 Key: YARN-4224
>                 URL: https://issues.apache.org/jira/browse/YARN-4224
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-4224-YARN-2928.01.patch, YARN-4224-feature-YARN-2928.wip.02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message