hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5585) [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters
Date Tue, 03 Jan 2017 05:58:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15794240#comment-15794240

Varun Saxena commented on YARN-5585:

Thanks [~rohithsharma] for the patch. Really sorry for the delay in reviewing. Few nits in

# In javadoc below, we say "prefix greater than or equal to and including the specified fromIdPrefix".
We can instead say "prefix greater than or equal to the specified fromIdPrefix" as greater
than or equal is inclusive of fromIdPrefix
102	 * <li><b>fromIdPrefix</b> - If specified, then retrieve entities with
an id
103	 * prefix greater than or equal to and including the specified fromIdPrefix. If
104	 * it is null, retrieved entities are always first 'N' rows in the back end
105	 * storage where N is limit.</li>
# The javadoc for fromId can be a little bit more descriptive. Just saying fromId is same
as entity ID doesn't seem to reflect what the filter does. We can explain that entities from
and after the combination of id prefix in fromidprefix and entity id carried in fromid, will
be returned.
419	   * @param fromId If specified, then fromIdPrefix is mandatory otherwise
420	   *     fromId is not considered as filter. fromId is same as entityId.
# Also the javadoc for entityidprefix just says "If specified, then entity retrieval is faster.".
Even though the variable is self explanatory but we can still say what it specifies i.e. it
specifies the id prefix for the entity to be fetched.

By the way as fromId is inclusive, we are expecting clients to find out the next ID in a pagination
scenario. Right ? We should probably document it once we do documentation for this part. I
have raised YARN-6047 to track documentation changes and this can be completed before 2nd
drop to trunk.

> [Atsv2] Reader side changes for entity prefix and support for pagination via additional
> -----------------------------------------------------------------------------------------------
>                 Key: YARN-5585
>                 URL: https://issues.apache.org/jira/browse/YARN-5585
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Critical
>              Labels: yarn-5355-merge-blocker
>         Attachments: 0001-YARN-5585.patch, YARN-5585-YARN-5355.0001.patch, YARN-5585-YARN-5355.0002.patch,
YARN-5585-YARN-5355.0003.patch, YARN-5585-YARN-5355.0004.patch, YARN-5585-YARN-5355.0005.patch,
YARN-5585-workaround.patch, YARN-5585.v0.patch
> TimelineReader REST API's provides lot of filters to retrieve the applications. Along
with those, it would be good to add new filter i.e fromId so that entities can be retrieved
after the fromId. 
> Current Behavior : Default limit is set to 100. If there are 1000 entities then REST
call gives first/last 100 entities. How to retrieve next set of 100 entities i.e 101 to 200
OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is no way
to achieve this. 
> So proposal is to have fromId in the filter like *getApps?limit=5&&fromId=app-5*
which gives list of apps from app-6 to app-10. 
> Since ATS is targeting large number of entities storage, it is very common use case to
get next set of entities using fromId rather than querying all the entites. This is very useful
for pagination in web UI.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message