hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5739) Provide timeline reader API to list available timeline entity types for one application
Date Tue, 08 Nov 2016 23:21:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649165#comment-15649165

Vrushali C commented on YARN-5739:

Thanks [~gtCarrera9] for the patch. I wish to add to [~varun_saxena]'s review suggestions.

- agree with suggestion to rename the rest endpoint, but I think we should not use "-" in
the rest endpoint string. So perhaps something like {quote} /apps/{appid}/entitytypes {quote}
- Yes we need not set max versions at L159 in EntityTypeReader
- WRT caching, I am wondering, if there might be a query coming next for the details of these
entity types?

For the scan/filter, I have a different suggestion:

Looks like we want to return only entity types. Entity types are part of row keys, we don't
need the column qualifiers and values in that case. So we can consider using the KeyOnlyFilter
filter.   This is a filter that will only return the key component of each KV (the value will
be rewritten as empty). This filter can be used to grab all of the keys without having to
also grab the values.  When performing a table scan where only the row keys are needed (no
families, qualifiers, values or timestamps), to use this, add a FilterList with a MUST_PASS_ALL
operator to the scanner using setFilter. The filter list should include both a FirstKeyOnlyFilter
and a KeyOnlyFilter. 

> Provide timeline reader API to list available timeline entity types for one application
> ---------------------------------------------------------------------------------------
>                 Key: YARN-5739
>                 URL: https://issues.apache.org/jira/browse/YARN-5739
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelinereader
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-5739-YARN-5355.001.patch
> Right now we only show a part of available timeline entity data in the new YARN UI. However,
some data (especially library specific data) are not possible to be queried out by the web
UI. It will be appealing for the UI to provide an "entity browser" for each YARN application.
Actually, simply dumping out available timeline entities (with proper pagination, of course)
would be pretty helpful for UI users. 
> On timeline side, we're not far away from this goal. Right now I believe the only thing
missing is to list all available entity types within one application. The challenge here is
that we're not storing this data for each application, but given this kind of call is relatively
rare (compare to writes and updates) we can perform some scanning during the read time. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message