hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5585) [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters
Date Tue, 03 Jan 2017 18:24:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15795729#comment-15795729

Rohith Sharma K S commented on YARN-5585:

bq. It is regarding the case where entity id prefix is not used for certain types of entities.
In that case, what do we say users should specify to accomplish pagination? Should they do
fromIdPrefix = 0 (default) and the fromId, or should they not specify fromIdPrefix at all?
There are 2 cases. 
# When idPrefix is incremental for entities of given entityType : Here it is known that entities
are sorted using id prefix. In this cases, ideally fromId is not mandatorily required to achieve
pagination. fromIdPrefix itself is sufficient to achieve pagination. 
# When idPrefix is same for all entities of given entityType(lets say 0) : Here, since idPrefix
is same across the entities, entities are sorted based on back end storage. In Hbase, it is
lexicographical order. But still, we need to achieve pagination. So, fromIdPrefix alone is
not sufficient, but also requires fromId. So, instead of allowing user to provide only fromId,
better enforce to provide fromIdPrefix and fromId. Note that we can NOT take default value
0 for fromIdPrefix if fromIdprefix is null because idPrefix is controlled by user.

> [Atsv2] Reader side changes for entity prefix and support for pagination via additional
> -----------------------------------------------------------------------------------------------
>                 Key: YARN-5585
>                 URL: https://issues.apache.org/jira/browse/YARN-5585
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Critical
>              Labels: yarn-5355-merge-blocker
>         Attachments: 0001-YARN-5585.patch, YARN-5585-YARN-5355.0001.patch, YARN-5585-YARN-5355.0002.patch,
YARN-5585-YARN-5355.0003.patch, YARN-5585-YARN-5355.0004.patch, YARN-5585-YARN-5355.0005.patch,
YARN-5585-workaround.patch, YARN-5585.v0.patch
> TimelineReader REST API's provides lot of filters to retrieve the applications. Along
with those, it would be good to add new filter i.e fromId so that entities can be retrieved
after the fromId. 
> Current Behavior : Default limit is set to 100. If there are 1000 entities then REST
call gives first/last 100 entities. How to retrieve next set of 100 entities i.e 101 to 200
OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is no way
to achieve this. 
> So proposal is to have fromId in the filter like *getApps?limit=5&&fromId=app-5*
which gives list of apps from app-6 to app-10. 
> Since ATS is targeting large number of entities storage, it is very common use case to
get next set of entities using fromId rather than querying all the entites. This is very useful
for pagination in web UI.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message