hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6861) Reader API for sub application entities
Date Sat, 29 Jul 2017 17:33:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16106187#comment-16106187
] 

Rohith Sharma K S commented on YARN-6861:
-----------------------------------------

Thanks [~vrushalic] for reviews. Some of the coding I was done vaguely to do POC. I will be
refactoring many of the stuff to reuse GenericEntitiesReader in next patch.
Mainly I wanted to get concession on REST APIs. I will exposing 2 REST API such as
# {{/entities/$entitytype}} : Returns collections
# {{/entities/$entitytype/$entity-id}} : Returns collections. Note this will not be returning
single entity because of table schema where same entity<type,id> can be exist across
the applications submitted user.

bq. Could we add a comment above the function saying this REST endpoint returns sub application
entities?
This is a important point to discuss. How should these APIs to be named? I mean we already
have generic entities which gets data from entity table. Should we keep it under Generic entities
itself which is another form of reading data? Or any specific different name should be given
to it?

bq. If I am understanding correctly, if the doAsUser is set in the context, this fetches the
sub application entities, else it will return the Generic entity reader in TimelineEntityReaderFactory?
Yes, doAsUser has set in context only from newly exposed APIs. For other API's this will be
null. For other API's, we do not really worry about doAsUser. If they are invoking those API's
it means it has different reader context.

bq. I am not sure if this is a very clean way. I am wondering how will this work from a browser
if I as a user want to fetch sub app entities for another user, say Varun?
We don't allow to read other doAsUser data unless request is coming from corresponding doAsUser.
Why we should allow other users data to be read? This breaks our basic assumption of ACLs
right? 

bq. Perhaps just accept the sub app user id as a Query param just like we accept userId? Also
perhaps we can call the end point something else so that it’s clear that we have sub application
entities not regular entities?
This is good idea to implement but my concern is we should also support URIs without any filters.
For other rest APIs, userid is filter which can be supplemented along with flowname and flowrunid.
Otherwise it will be fetched from appToFlow table. But for subAppEntities, we don't have any
such indexing to look up. Filters can not be mandatory. In such cases always, scanning need
to be done begging of table nevertheless of any user. 

> Reader API for sub application entities
> ---------------------------------------
>
>                 Key: YARN-6861
>                 URL: https://issues.apache.org/jira/browse/YARN-6861
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>         Attachments: YARN-6861-YARN-5355.001.patch
>
>
> YARN-6733 and YARN-6734 writes data into sub application table. There should be a way
to read those entities.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message