hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3862) Decide which contents to retrieve and send back in response in TimelineReader
Date Tue, 03 Nov 2015 01:17:27 GMT

    [ https://issues.apache.org/jira/browse/YARN-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986444#comment-14986444
] 

Sangjin Lee commented on YARN-3862:
-----------------------------------

I just kicked off a jenkins run on the latest patch.

Sorry [~varun_saxena] it took me a while to get around to looking at the patch. The overall
approach seems pretty reasonable to me. I'll need to go over the patch in some detail, however.

One point I'd like to make is regarding {{TimelineEntityReader.constructFilterListBasedOnFields()}}.
I see it using {{EntityColumnFamily}} and {{EntityColumnPrefix}}. I don't think that's quite
right. In terms of the class hierarchy {{TimelineEntityReader}} sits below {{GenericEntityReader}}
(which deals with the generic entity table). As such, it should be agnostic to the actual
specific tables. The {{TimelineEntityReader.constructFilterListBasedOnFields()}} method should
contain only the most generic implementation (which may well be returning null). Any logic
that deals with the entity columns should belong in {{GenericEntityReader}}.

This also points to an issue with {{ApplicationEntityReader}}. Its {{constructFilterListBasedOnFields()}}
method needs to be implemented in terms of application column family and application column
prefix. So it needs to be properly overridden in that class.

> Decide which contents to retrieve and send back in response in TimelineReader
> -----------------------------------------------------------------------------
>
>                 Key: YARN-3862
>                 URL: https://issues.apache.org/jira/browse/YARN-3862
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-3862-YARN-2928.wip.01.patch, YARN-3862-YARN-2928.wip.02.patch
>
>
> Currently, we will retrieve all the contents of the field if that field is specified
in the query API. In case of configs and metrics, this can become a lot of data even though
the user doesn't need it. So we need to provide a way to query only a set of configs or metrics.
> As a comma spearated list of configs/metrics to be returned will be quite cumbersome
to specify, we have to support either of the following options :
> # Prefix match
> # Regex
> # Group the configs/metrics and query that group.
> We also need a facility to specify a metric time window to return metrics in a that window.
This may be useful in plotting graphs 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message