hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers
Date Wed, 10 Jun 2015 17:36:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580858#comment-14580858

Varun Saxena commented on YARN-3051:

[~zjshen], thanks for your inputs. I will brief you about the APIs' I have decided as of now.

# APIs' for querying individual entity/flow/flow run/user and APIs' for querying a set of
entities/flow runs/flows/users. APIs' such a set of flows/users will contain aggregated data.
The reason for separate endpoints for entities, flows, users,etc. is because of the different
tables in HBase/Phoenix schema.
# Most the APIs' will be variations of either getting a single entity or a set of entities.
So I will primarily talk about entity and a set of entities in subsequent points.
# For getting a set of entities, there will be 3 kinds of filters - filtering on the basis
of info, filtering on configs and filtering on metrics. Filtering on the basis of info and
field will be based on equality, for instance, fetch entities which have a config name matching
a specific config value. Metrics filtering though will be on the basis of relational operator.
For instance, user can query entities which have a specific metric >= a certain value.
# In addition to that certain predicates such as limit, windowStart, windowEnd, etc. which
used to exist in ATSv1 exist even now.Some predicates such as fromId, fromTs may not make
sense in ATSv2 but I have included them for now with the intention of discussion.
# Additional predicates such as metricswindowStart and end has been specified to fetch metrics
data for a specific time span. The reason I included this is because this can aid in plotting
graphs on UI for a specific metric of some entity.
# Only entity id, type, created and modified time will be returned if fields are not specified
in REST URL. This will be the default view of an entity.
# Moreover you can also specify which configurations and metrics to return.
# Every query param will be received as a String, even timestamp. Now from backing storage
implementation viewpoint, would it make more sense to let these query params be passed as
strings or do datatype conversion ?

Few concerns from Li Lu regarding parameter list becoming too long are quite valid as most
of them will be nulls. We can also club multiple related parameters in a different classes
to reduce them. Or as he said have different methods for frequently occurring use cases. Thoughts

Comments are welcome so that this JIRA can speed up, probably after Hadoop Summit :)

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051-YARN-2928.003.patch, YARN-3051-YARN-2928.03.patch, YARN-3051.wip.02.YARN-2928.patch,
YARN-3051.wip.patch, YARN-3051_temp.patch
> Per design in YARN-2928, create backing storage read interface that can be implemented
by multiple backing storage implementations.

This message was sent by Atlassian JIRA

View raw message