hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers
Date Tue, 31 Mar 2015 16:00:57 GMT

    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388748#comment-14388748
] 

Varun Saxena commented on YARN-3051:
------------------------------------

I will explain the APIs' added and use cases it will cover :
      {code}TimelineEntities getEntities(String entityType, long limit,
      long windowStart, long windowEnd, String fromId, long fromTs,
      Collection<NameValuePair> filters,
      Collection<NameValuePair> confFilters,
      Collection<NameValueRelation> metricFilters,
      Collection<String> confs, Collection<String> metrics,
      EnumSet<Field> fieldsToRetrieve) throws IOException; {code}
This API returns a set of entities based on entity type. The params passed are as follows:
#  *limit* - Number of entities to be returned.
# *windowStart*, *windowEnd* - Decides the time window for entities to be returned. Uses created
time. Modified time to be used ?
# *filters* - Similar to secondary filters. Will match info field in timeline entity. 
# *confFilters* - Similar to filters above. Will match configs.
# *metricFilters* - Filter entities based on metric values. Would support all relational operators
(=,>,<=,<,<=,!=). In case of timeseries data, we can match it with the latest
record. Thoughts ? This covers queries such as get apps where metricX >= Y. 
# *confs* - Specifies configurations to be returned(comma separated list).
# *metrics* - Specifies metrics to be returned(comma separated list)
# *fieldsToRetrieve* - Specifies which fields to retrieve. As discussed above, default view
of entity will  be returned if fields are not specified. This has to be taken care by store
implementation. Would mentioning it in javadoc be enough ? Or TimelineReaderDataManager can
fill the fieldsToRetrieve EnumSet to indicate to store which specific fields have to be returned(even
the default ones).

{code}
  TimelineEntity getEntity(String entityId, String entityType,
      Collection<String> confs, Collection<String> metrics,
      EnumSet<Field> fieldsToRetrieve) throws IOException;
{code]
Similar to above. Gets a single entity based on entityid and entitytype. Need to discuss how
cluster id, flow id, flow run id,  etc. would impact this API.

{code}

  TimelineEvents getEntityTimelines(String entityType,
      SortedSet<String> entityIds, long limit, long windowStart,
      long windowEnd, Set<String> eventTypes) throws IOException;
{code}
This API gets list of events for a list of entities for a given entity type. The event types
can also be restricted. Carried over from ATS v1.

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051_temp.patch
>
>
> Per design in YARN-2928, create backing storage read interface that can be implemented
by multiple backing storage implementations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message