Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Mon, 9 Nov 2015 19:09:11 +0000 (UTC)
From: "Varun Saxena (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12841288.1435573666000.18881.1447096151132@Atlassian.JIRA>
In-Reply-To: <JIRA.12841288.1435573666000@Atlassian.JIRA>
References: <JIRA.12841288.1435573666000@Atlassian.JIRA>
 <JIRA.12841288.1435573666087@arcas>
Subject: [jira] [Commented] (YARN-3862) Decide which contents to retrieve
 and send back in response in TimelineReader
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997161#comment-14997161 ] 

Varun Saxena commented on YARN-3862:
------------------------------------

Attached a WIP patch.

This patch attempts to do handling for all the tables(creation of filter list based on fields) and do prefix matching for configs and filters. Previous WIP patch was only attempting to handle for entity table because of the implications of this patch on config and metric filters' matching. Have handled this scenario in this patch. 
When YARN-3863 is done, some changes will be warranted though(some conditions to pass config and metric filters will have to be removed). 
Have added a few tests to test the change as well.

I have still not hooked up this code to REST API layer.
For that, we first need to decide as to whether the TimelineFilter code will be part of our object model or not.
For prefix matching of configs and metrics to return, at the REST layer this can simply come as a query param (a comma separated list)

But when we code for complex filters (especially metric filters) in YARN-3863 we will have to support SQL type queries with ANDs', ORs', >,<,=operators, etc.
If we make TimelineFilter as part of our client object model and interpret filters as a JSON string associated with a query param, we might have to rethink a bit on few of the classes and  including additional checks(as this will be used by client).
This can increase size of the URL though.

If we do not include filter as part of our object model, we will have to decide how to specify complex config and metric filters containing ANDs' and ORs' and different relational operators(because some of the symbols will be reserved) and reach a consensus on that.


> Decide which contents to retrieve and send back in response in TimelineReader
> -----------------------------------------------------------------------------
>
>                 Key: YARN-3862
>                 URL: https://issues.apache.org/jira/browse/YARN-3862
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-3862-YARN-2928.wip.01.patch, YARN-3862-YARN-2928.wip.02.patch, YARN-3862-YARN-2928.wip.03.patch
>
>
> Currently, we will retrieve all the contents of the field if that field is specified in the query API. In case of configs and metrics, this can become a lot of data even though the user doesn't need it. So we need to provide a way to query only a set of configs or metrics.
> As a comma spearated list of configs/metrics to be returned will be quite cumbersome to specify, we have to support either of the following options :
> # Prefix match
> # Regex
> # Group the configs/metrics and query that group.
> We also need a facility to specify a metric time window to return metrics in a that window. This may be useful in plotting graphs 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)