hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-5585) [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters
Date Tue, 03 Jan 2017 16:14:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15795388#comment-15795388
] 

Varun Saxena edited comment on YARN-5585 at 1/3/17 4:14 PM:
------------------------------------------------------------

Right. Let's first reach a consensus on it as it's merely a few changes in sentences in javadoc.

Taking a cue from what you mentioned above, I propose the following.
For fromIdPrefix
{code}
If specified, retrieve entities with an id prefix greater than or equal to the specified fromIdPrefix.
If fromIdPrefix is same for all entities of a given entity type, then the user must provide
fromId as a filter to denote the start entity from which further entities will be fetched.
{code}

For fromId
{code}
If specified along with fromIdPrefix, retrieve entities with an id prefix greater than or
equal to specified id prefix in fromIdPrefix and entity id lexicographically greater than
or equal to entity id specified in fromId. Please note than fromIdPrefix is mandatory if fromId
is specified, otherwise, the filter will be ignored. It is recommended to provide both fromIdPrefix
and fromId filters for more accurate results as id prefix may not be unique for an entity.
{code}

For entityidprefix
{code}
Defines the id prefix for the entity to be fetched. If specified, then entity retrieval will
be faster.
{code}

How does this sound ?

I think other than these small nits, the patch is good to go. Once this is fixed, I will commit
it in a couple of days. That's because other guys in the team would have now come back from
holidays and they can probably have a look as well.


was (Author: varun_saxena):
Right lets first reach a consensus as its merely a few changes in sentences in javadoc.

Taking cue from what you mentioned above, I propose the following.
For fromIdPrefix
{code}
If specified, retrieve entities with an id prefix greater than or equal to the specified fromIdPrefix.
If fromIdPrefix is same for all entities in a given entity type then user must provide fromId
as a filter to denote the start entity from which further entities will be fetched.
{code}

For fromId
{code}
If specified alongwith fromIdPrefix, retrieve entities with an id prefix greater than or equal
to specified id prefix in fromIdPrefix and entity id lexicographically greater than or equal
to entity id specified in fromId. Please note than fromIdPrefix is mandatory if fromId is
specified, otherwise the filter will be ignored. It is recommended to provide both fromIdPrefix
and fromId filters for more accurate results as id prefix may not be unique for an entity.
{code}

For entityidprefix
{code}
Defines the id prefix for the entity to be fetched. If specified, then entity retrieval will
be faster.
{code}

How does this sound ?

I think other than these small nits, the patch is good to go. Once this is fixed, I will commit
it in a couple of days. That's because other guys in the team would have now come back from
holidays and they can probably have a look as well.

> [Atsv2] Reader side changes for entity prefix and support for pagination via additional
filters
> -----------------------------------------------------------------------------------------------
>
>                 Key: YARN-5585
>                 URL: https://issues.apache.org/jira/browse/YARN-5585
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Critical
>              Labels: yarn-5355-merge-blocker
>         Attachments: 0001-YARN-5585.patch, YARN-5585-YARN-5355.0001.patch, YARN-5585-YARN-5355.0002.patch,
YARN-5585-YARN-5355.0003.patch, YARN-5585-YARN-5355.0004.patch, YARN-5585-YARN-5355.0005.patch,
YARN-5585-workaround.patch, YARN-5585.v0.patch
>
>
> TimelineReader REST API's provides lot of filters to retrieve the applications. Along
with those, it would be good to add new filter i.e fromId so that entities can be retrieved
after the fromId. 
> Current Behavior : Default limit is set to 100. If there are 1000 entities then REST
call gives first/last 100 entities. How to retrieve next set of 100 entities i.e 101 to 200
OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is no way
to achieve this. 
> So proposal is to have fromId in the filter like *getApps?limit=5&&fromId=app-5*
which gives list of apps from app-6 to app-10. 
> Since ATS is targeting large number of entities storage, it is very common use case to
get next set of entities using fromId rather than querying all the entites. This is very useful
for pagination in web UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message