Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 6 Oct 2016 17:29:20 +0000 (UTC)
From: "Varun Saxena (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13001293.1472562035000.755218.1475774960662@Atlassian.JIRA>
In-Reply-To: <JIRA.13001293.1472562035000@Atlassian.JIRA>
References: <JIRA.13001293.1472562035000@Atlassian.JIRA> <JIRA.13001293.1472562035465@arcas>
Subject: [jira] [Commented] (YARN-5585) [Atsv2] Add a new filter fromId in
 REST endpoints
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 06 Oct 2016 17:29:22 -0000


    [ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15552593#comment-15552593 ] 

Varun Saxena commented on YARN-5585:
------------------------------------

bq. Given entities are sorted in ascending order, at some extent latest fist order can be achieve by doing reverse scan. I had tried this for yarn-containers and works fine.
Reverse scan would work fine but how do we decide which entity type would need it and which won't. By the way we need container IDs' in the reverse order too ? IIRC, in one of the calls Li mentioned lexicographic order should be fine for new Web UI. If required we can have special handling for YARN specific entities like App attempts and Containers, just like we have for apps.
No matter what we do, it should be consistent across all entities. We can also have another query param to indicate reverse lexicographic order is required.

bq. IIUC, AM can delegate collector address to any of its running containers to publish its own data. TimelineClient can not be restricted to only AM.
True. In a secure setup, AM can even pass on the token. The point is we support talking to AM only. AM can then delegate its work to anyone. But the concern here was that prefix will have to be passed around by AM via a new protocol. So if application wants to support delegating work to other processes, it anyways needs to open new protocol. So I guess this concern is not specific to prefix. Correct ? However, would be useful if you can tell the use case of multiple JVMs'. Same DAGs' can be executed by different processes. This would help us thing 

> [Atsv2] Add a new filter fromId in REST endpoints
> -------------------------------------------------
>
>                 Key: YARN-5585
>                 URL: https://issues.apache.org/jira/browse/YARN-5585
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Rohith Sharma K S
>            Assignee: Rohith Sharma K S
>            Priority: Critical
>         Attachments: 0001-YARN-5585.patch, YARN-5585-workaround.patch, YARN-5585.v0.patch
>
>
> TimelineReader REST API's provides lot of filters to retrieve the applications. Along with those, it would be good to add new filter i.e fromId so that entities can be retrieved after the fromId. 
> Current Behavior : Default limit is set to 100. If there are 1000 entities then REST call gives first/last 100 entities. How to retrieve next set of 100 entities i.e 101 to 200 OR 900 to 801?
> Example : If applications are stored database, app-1 app-2 ... app-10.
> *getApps?limit=5* gives app-1 to app-5. But to retrieve next 5 apps, there is no way to achieve this. 
> So proposal is to have fromId in the filter like *getApps?limit=5&&fromId=app-5* which gives list of apps from app-6 to app-10. 
> Since ATS is targeting large number of entities storage, it is very common use case to get next set of entities using fromId rather than querying all the entites. This is very useful for pagination in web UI.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org