hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5585) [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters
Date Mon, 19 Dec 2016 23:56:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15762691#comment-15762691
] 

Li Lu commented on YARN-5585:
-----------------------------

Thanks [~rohithsharma] for the update! With regards to the APIs, I think we can pretty much
reuse the current set of APIs. IMO, we should not force a prefix all the time. Of course,
if the user knows the exact entity prefix it's certainly beneficial to include it in the query
(so that we can save a range scan and just use a get). When referring to timeline entity ids,
how about the following patterns:
1. <string1>!<string2>: string 1 is the prefix and string 2 is the id
2. <string> or *\!<string>: string is the entity id and the storage needs to query
the entity prefix. If we have problems distinguishing <string> from the above case maybe
we can use *\!<string>

bq. If we plan to reuse same API's, then we need to handle one scenario where same entityId
is published with 2 entityIdPrefix.
This sounds like a really messy situation. Semantically, we've got two ways to decide this:
1) we explicitly claim that entity prefix id is a part of the id system. This means two entities
are different even if they only only differ in entity prefixes and 2)  we claim that entity
prefix is _not_ a part of the id system. Under this assumption, it is up to the storage system
to decide how to deal the case which prefixes are updated. Therefore the behavior when one
entity is associated with two prefixes, from the API level, is undefined. As [~varun_saxena]
suggested, the storage may throw exceptions or return errors when multiple prefixes are found
for the same entity. 

> [Atsv2] Reader side changes for entity prefix and support for pagination via additional
filters
> -----------------------------------------------------------------------------------------------
>
>                 Key: YARN-5585
>                 URL: https://issues.apache.org/jira/browse/YARN-5585
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Critical
>              Labels: yarn-5355-merge-blocker
>         Attachments: 0001-YARN-5585.patch, YARN-5585-YARN-5355.0001.patch, YARN-5585-YARN-5355.0002.patch,
YARN-5585-workaround.patch, YARN-5585.v0.patch
>
>
> TimelineReader REST API's provides lot of filters to retrieve the applications. Along
with those, it would be good to add new filter i.e fromId so that entities can be retrieved
after the fromId. 
> Current Behavior : Default limit is set to 100. If there are 1000 entities then REST
call gives first/last 100 entities. How to retrieve next set of 100 entities i.e 101 to 200
OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is no way
to achieve this. 
> So proposal is to have fromId in the filter like *getApps?limit=5&&fromId=app-5*
which gives list of apps from app-6 to app-10. 
> Since ATS is targeting large number of entities storage, it is very common use case to
get next set of entities using fromId rather than querying all the entites. This is very useful
for pagination in web UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message