Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 24 Nov 2016 20:20:58 +0000 (UTC)
From: "Varun Saxena (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13012490.1476484616000.363473.1480018858455@Atlassian.JIRA>
In-Reply-To: <JIRA.13012490.1476484616000@Atlassian.JIRA>
References: <JIRA.13012490.1476484616000@Atlassian.JIRA> <JIRA.13012490.1476484616419@arcas>
Subject: [jira] [Commented] (YARN-5739) Provide timeline reader API to list
 available timeline entity types for one application
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 24 Nov 2016 20:21:00 -0000


    [ https://issues.apache.org/jira/browse/YARN-5739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15694139#comment-15694139 ] 

Varun Saxena commented on YARN-5739:
------------------------------------

I was wondering if we can refactor this code. As I said when I first reviewed this JIRA, EntityTypeReader being a subclass of GenericEntityReader does not seem correct. Infact is it even fit to be a subclass of TimelineEntityReader ? We are not attempting to return timeline entities here however it can be said in backend we store a timeline entity only. 
Basically because we are deriving EntityTypeReader from TimelineEntityReader we have to override readEntities and create the response as a set of  TimelineEntity objects unnecessarily even though what we just need is a list of entity types. I just find this part a little weird.

Looking at the code, the main thing which we need from GenericEntityReader is the part about looking up flow context. Rest of the required code is very small. Should we pull out the code related to AppToFlowTable querying (i.e. looking up flow context) and move it to a separate class ? 
We can then have EntityTypeReader as a standalone class with both it and GenericEntityReader referring to another class to look up flow context.

I do not think when it comes to querying entity types, even the filters or timeline data to retrieve stored in TimelineEntityReader will be useful.
The best I can think of supporting as a filter for this query is that we may want to return entity types starting with a certain prefix.

Thoughts ?


> Provide timeline reader API to list available timeline entity types for one application
> ---------------------------------------------------------------------------------------
>
>                 Key: YARN-5739
>                 URL: https://issues.apache.org/jira/browse/YARN-5739
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-5739-YARN-5355.001.patch, YARN-5739-YARN-5355.002.patch, YARN-5739-YARN-5355.003.patch, YARN-5739-YARN-5355.004.patch
>
>
> Right now we only show a part of available timeline entity data in the new YARN UI. However, some data (especially library specific data) are not possible to be queried out by the web UI. It will be appealing for the UI to provide an "entity browser" for each YARN application. Actually, simply dumping out available timeline entities (with proper pagination, of course) would be pretty helpful for UI users. 
> On timeline side, we're not far away from this goal. Right now I believe the only thing missing is to list all available entity types within one application. The challenge here is that we're not storing this data for each application, but given this kind of call is relatively rare (compare to writes and updates) we can perform some scanning during the read time. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org