Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 20 Dec 2016 19:24:58 +0000 (UTC)
From: "Varun Saxena (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13001293.1472562035000.565576.1482261898711@Atlassian.JIRA>
In-Reply-To: <JIRA.13001293.1472562035000@Atlassian.JIRA>
References: <JIRA.13001293.1472562035000@Atlassian.JIRA> <JIRA.13001293.1472562035465@arcas>
Subject: [jira] [Commented] (YARN-5585) [Atsv2] Reader side changes for
 entity prefix and support for pagination via additional filters
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 20 Dec 2016 19:25:00 -0000


    [ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765025#comment-15765025 ] 

Varun Saxena commented on YARN-5585:
------------------------------------

Thanks Rohith for the patch.

# Code for splitting fromId as it is repeated can be moved to a method in TimelineReaderWebServicesUtils and we can call it after creating TimelineReaderContext object to populate relevant fields.
# Is there any need to populate id prefix in TimelineReaderManager#fillUID
# Add a javadoc for the new query params.
# As all our query params are in lower case we can name it as "fromid" ?
# TimelineFilterUtils#createSingleColValueFilters doesnt seem like an apt name as it returns a single filter. Also why wrap a single filter in a filter list ? We can probably call createHBaseSingleColValue method directly from GenericEntityReader#getResult
# In getResult how about setting a PageFilter of 2 in addition to SingleColumnValueFilter ? This will reduce the rows scanned if there are many duplicate rows. Not a typical case though.
# GenericEntityReader line 458 there is a typo. Should not be idprefixe
# Should we throw an exception for get entities call too if duplicate entity is found ?


> [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters
> -----------------------------------------------------------------------------------------------
>
>                 Key: YARN-5585
>                 URL: https://issues.apache.org/jira/browse/YARN-5585
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Critical
>              Labels: yarn-5355-merge-blocker
>         Attachments: 0001-YARN-5585.patch, YARN-5585-YARN-5355.0001.patch, YARN-5585-YARN-5355.0002.patch, YARN-5585-YARN-5355.0003.patch, YARN-5585-workaround.patch, YARN-5585.v0.patch
>
>
> TimelineReader REST API's provides lot of filters to retrieve the applications. Along with those, it would be good to add new filter i.e fromId so that entities can be retrieved after the fromId. 
> Current Behavior : Default limit is set to 100. If there are 1000 entities then REST call gives first/last 100 entities. How to retrieve next set of 100 entities i.e 101 to 200 OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is no way to achieve this. 
> So proposal is to have fromId in the filter like *getApps?limit=5&&fromId=app-5* which gives list of apps from app-6 to app-10. 
> Since ATS is targeting large number of entities storage, it is very common use case to get next set of entities using fromId rather than querying all the entites. This is very useful for pagination in web UI.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org