hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5585) [Atsv2] Add a new filter fromId in REST endpoints
Date Wed, 07 Sep 2016 14:09:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15470739#comment-15470739

Varun Saxena commented on YARN-5585:

bq. But problem is entities may not stored in their creation time order.  Say, if entities
are stored in a row <EntityId-3, T1> <EntityId-1, T2> <EntityId-2, T3> then
the above approach would fail. Because say fromId=Entity-2 expecting to get Entity-3 but createtime
is T3 which empty rows are retrieved from HBase.
Not really. From the description of fromId in ATSv1, "If fromId is not null, retrieve entities
earlier than and including the specified ID. If no start time is found for the specified ID,
an empty list of entities will be returned. ". We wont be specifying entity id in row key.
I am not suggesting to specify this as stop row. So its not as if  we will start or stop at
a specific row.
The solution suggested above still means we will parse each and every entity row within the
scope of entity type and compare against created time.

This is not a great solution because we are still having a large scan but better than fetching
everything back to client. I will propose some other solution.

> [Atsv2] Add a new filter fromId in REST endpoints
> -------------------------------------------------
>                 Key: YARN-5585
>                 URL: https://issues.apache.org/jira/browse/YARN-5585
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>         Attachments: YARN-5585.v0.patch
> TimelineReader REST API's provides lot of filters to retrieve the applications. Along
with those, it would be good to add new filter i.e fromId so that entities can be retrieved
after the fromId. 
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-10. But to retrieve next 5 apps, it is difficult.
> So proposal is to have fromId in the filter like *getApps?limit=5&&fromId=app-5*
which gives list of apps from app-6 to app-10. 
> This is very useful for pagination in web UI.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message