hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3862) Decide which contents to retrieve and send back in response in TimelineReader
Date Mon, 09 Nov 2015 19:09:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997161#comment-14997161

Varun Saxena commented on YARN-3862:

Attached a WIP patch.

This patch attempts to do handling for all the tables(creation of filter list based on fields)
and do prefix matching for configs and filters. Previous WIP patch was only attempting to
handle for entity table because of the implications of this patch on config and metric filters'
matching. Have handled this scenario in this patch. 
When YARN-3863 is done, some changes will be warranted though(some conditions to pass config
and metric filters will have to be removed). 
Have added a few tests to test the change as well.

I have still not hooked up this code to REST API layer.
For that, we first need to decide as to whether the TimelineFilter code will be part of our
object model or not.
For prefix matching of configs and metrics to return, at the REST layer this can simply come
as a query param (a comma separated list)

But when we code for complex filters (especially metric filters) in YARN-3863 we will have
to support SQL type queries with ANDs', ORs', >,<,=operators, etc.
If we make TimelineFilter as part of our client object model and interpret filters as a JSON
string associated with a query param, we might have to rethink a bit on few of the classes
and  including additional checks(as this will be used by client).
This can increase size of the URL though.

If we do not include filter as part of our object model, we will have to decide how to specify
complex config and metric filters containing ANDs' and ORs' and different relational operators(because
some of the symbols will be reserved) and reach a consensus on that.

> Decide which contents to retrieve and send back in response in TimelineReader
> -----------------------------------------------------------------------------
>                 Key: YARN-3862
>                 URL: https://issues.apache.org/jira/browse/YARN-3862
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-3862-YARN-2928.wip.01.patch, YARN-3862-YARN-2928.wip.02.patch,
> Currently, we will retrieve all the contents of the field if that field is specified
in the query API. In case of configs and metrics, this can become a lot of data even though
the user doesn't need it. So we need to provide a way to query only a set of configs or metrics.
> As a comma spearated list of configs/metrics to be returned will be quite cumbersome
to specify, we have to support either of the following options :
> # Prefix match
> # Regex
> # Group the configs/metrics and query that group.
> We also need a facility to specify a metric time window to return metrics in a that window.
This may be useful in plotting graphs 

This message was sent by Atlassian JIRA

View raw message