hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5585) [Atsv2] Add a new filter fromId in REST endpoints
Date Thu, 29 Sep 2016 19:39:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15533830#comment-15533830
] 

Vrushali C commented on YARN-5585:
----------------------------------

Thanks [~rohithsharma] for the summary.

bq. 2. By default, use createdTime as entityPrefixId.
Also, that means, frameworks which don't want to use the entity id prefix have to explicitly
specify a null prefix (or a special value that means null).
All the same, it will be really good to mention in the docs for clients that they should do
the following. 

{code:title=TimelineWriterClient.java}
entity.setEntityPrefix(createdTime);
client.writeEntity(entity); // pseudo-code
{code}

bq. For the REST end point, we can support fromEntityPrefixId will become combination of entityPrefixId+entityId
which can be used for pagination
I think pagination handling should be more generic than depending on something like "fromEntityPrefixId".
 REST queries should simply ask for top N records with the understanding that the records
are returned in sorted order of entity prefixes. For the next page of results, the client
sends back the last row returned's key/entity prefix. For a rest query, if the "startFrom"
query param is present, the scan starts from "startFrom" prefix value and returns the next
N such records.




> [Atsv2] Add a new filter fromId in REST endpoints
> -------------------------------------------------
>
>                 Key: YARN-5585
>                 URL: https://issues.apache.org/jira/browse/YARN-5585
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Critical
>         Attachments: YARN-5585-workaround.patch, YARN-5585.v0.patch
>
>
> TimelineReader REST API's provides lot of filters to retrieve the applications. Along
with those, it would be good to add new filter i.e fromId so that entities can be retrieved
after the fromId. 
> Current Behavior : Default limit is set to 100. If there are 1000 entities then REST
call gives first/last 100 entities. How to retrieve next set of 100 entities i.e 101 to 200
OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is no way
to achieve this. 
> So proposal is to have fromId in the filter like *getApps?limit=5&&fromId=app-5*
which gives list of apps from app-6 to app-10. 
> Since ATS is targeting large number of entities storage, it is very common use case to
get next set of entities using fromId rather than querying all the entites. This is very useful
for pagination in web UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message