Return-Path: 
 <yarn-issues-return-52379-apmail-hadoop-yarn-issues-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 6A48417C51
	for <apmail-hadoop-yarn-issues-archive@minotaur.apache.org>;
 Tue, 31 Mar 2015 16:00:58 +0000 (UTC)
Received: (qmail 81165 invoked by uid 500); 31 Mar 2015 16:00:58 -0000
Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org
Received: (qmail 81108 invoked by uid 500); 31 Mar 2015 16:00:58 -0000
Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:yarn-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:yarn-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:yarn-issues@hadoop.apache.org>
List-Id: <yarn-issues.hadoop.apache.org>
Reply-To: yarn-issues@hadoop.apache.org
Delivered-To: mailing list yarn-issues@hadoop.apache.org
Received: (qmail 81015 invoked by uid 99); 31 Mar 2015 16:00:58 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 31 Mar 2015 16:00:58 +0000
Date: Tue, 31 Mar 2015 16:00:57 +0000 (UTC)
From: "Varun Saxena (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12766998.1421110823000.81773.1427817657979@Atlassian.JIRA>
In-Reply-To: <JIRA.12766998.1421110823000@Atlassian.JIRA>
References: <JIRA.12766998.1421110823000@Atlassian.JIRA>
 <JIRA.12766998.1421110823412@arcas>
Subject: [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing
 storage read interface for ATS readers
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388748#comment-14388748 ] 

Varun Saxena commented on YARN-3051:
------------------------------------

I will explain the APIs' added and use cases it will cover :
      {code}TimelineEntities getEntities(String entityType, long limit,
      long windowStart, long windowEnd, String fromId, long fromTs,
      Collection<NameValuePair> filters,
      Collection<NameValuePair> confFilters,
      Collection<NameValueRelation> metricFilters,
      Collection<String> confs, Collection<String> metrics,
      EnumSet<Field> fieldsToRetrieve) throws IOException; {code}
This API returns a set of entities based on entity type. The params passed are as follows:
#  *limit* - Number of entities to be returned.
# *windowStart*, *windowEnd* - Decides the time window for entities to be returned. Uses created time. Modified time to be used ?
# *filters* - Similar to secondary filters. Will match info field in timeline entity. 
# *confFilters* - Similar to filters above. Will match configs.
# *metricFilters* - Filter entities based on metric values. Would support all relational operators (=,>,<=,<,<=,!=). In case of timeseries data, we can match it with the latest record. Thoughts ? This covers queries such as get apps where metricX >= Y. 
# *confs* - Specifies configurations to be returned(comma separated list).
# *metrics* - Specifies metrics to be returned(comma separated list)
# *fieldsToRetrieve* - Specifies which fields to retrieve. As discussed above, default view of entity will  be returned if fields are not specified. This has to be taken care by store implementation. Would mentioning it in javadoc be enough ? Or TimelineReaderDataManager can fill the fieldsToRetrieve EnumSet to indicate to store which specific fields have to be returned(even the default ones).

{code}
  TimelineEntity getEntity(String entityId, String entityType,
      Collection<String> confs, Collection<String> metrics,
      EnumSet<Field> fieldsToRetrieve) throws IOException;
{code]
Similar to above. Gets a single entity based on entityid and entitytype. Need to discuss how cluster id, flow id, flow run id,  etc. would impact this API.

{code}

  TimelineEvents getEntityTimelines(String entityType,
      SortedSet<String> entityIds, long limit, long windowStart,
      long windowEnd, Set<String> eventTypes) throws IOException;
{code}
This API gets list of events for a list of entities for a given entity type. The event types can also be restricted. Carried over from ATS v1.

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051_temp.patch
>
>
> Per design in YARN-2928, create backing storage read interface that can be implemented by multiple backing storage implementations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)