hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5585) [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters
Date Thu, 22 Dec 2016 16:56:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15770520#comment-15770520

Varun Saxena commented on YARN-5585:

bq. More importantly, we have missed one point over here. 
That's a good point. But I guess this can be done in another JIRA. Because this primarily
deals with prefix handling and hence touches upon entity table. Otherwise changes would become
quite large.
Anyways, server side pagination is primarily useful for cases where a single call may retrieve
a substantial bunch of data. The use cases pointed out above will be useful for new YARN UI.
We should support this for apps within a flow. I see that as definitely being useful. As we
are doing it for apps within a flow, it can be done for apps within flowrun as well because
both touch app table.
Over a period, runs for a flow can also be substantial I guess. 
In web UI we will probably club flows by dates because its basically flow activity for a day.
So, date range filter which is already there can be used. We need to consider though if number
of flows for each day can be too much and would it be useful to retrieve it in batches within
the scope of a date. Probably not. 

bq. So, I think we can separate fromId as two parts
I am fine with this as well. Infact this was amongst one of the suggestions given earlier

> [Atsv2] Reader side changes for entity prefix and support for pagination via additional
> -----------------------------------------------------------------------------------------------
>                 Key: YARN-5585
>                 URL: https://issues.apache.org/jira/browse/YARN-5585
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Critical
>              Labels: yarn-5355-merge-blocker
>         Attachments: 0001-YARN-5585.patch, YARN-5585-YARN-5355.0001.patch, YARN-5585-YARN-5355.0002.patch,
YARN-5585-YARN-5355.0003.patch, YARN-5585-workaround.patch, YARN-5585.v0.patch
> TimelineReader REST API's provides lot of filters to retrieve the applications. Along
with those, it would be good to add new filter i.e fromId so that entities can be retrieved
after the fromId. 
> Current Behavior : Default limit is set to 100. If there are 1000 entities then REST
call gives first/last 100 entities. How to retrieve next set of 100 entities i.e 101 to 200
OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is no way
to achieve this. 
> So proposal is to have fromId in the filter like *getApps?limit=5&&fromId=app-5*
which gives list of apps from app-6 to app-10. 
> Since ATS is targeting large number of entities storage, it is very common use case to
get next set of entities using fromId rather than querying all the entites. This is very useful
for pagination in web UI.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message