hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3031) [Storage abstraction] Create backing storage write interface for ATS writers
Date Wed, 25 Feb 2015 04:51:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14336001#comment-14336001

Zhijie Shen commented on YARN-3031:

[~vrushalic], thanks for updating the patch! Some comments on the APIs:

1. According to YARN-3041, metric and event are neither the standalone objects, nor associated
to application entity only. For example, If I want to write the container-level metrics and
events, being given appId is not enough for me associate these objects to the container entity
they should belong to. IMHO, entity identifier is more appropriate instead of appId. 

TimelineServiceWriteResponse updateMetrics(ApplicationId appId,
    TimelineMetric metric) throws IOException;
TimelineServiceWriteResponse addEvent(ApplicationId appId,
     TimelineEvent data) throws IOException;

Think it out loudly, can we just have a single {{write(TimelineEntity entity)}}. To store
the metrics and events, we always use the entity as the holder them.

2. Perhaps we even don't need to have separate write and aggregate APIs. We can have a single
write API, but also take AggregateUpTo as the second param. If it is provided, the store impl
does the aggregation at the given dimension, otherwise, it simply persists the entity.

3. It seems that TimelineServiceWriteResponse is copied from the old TimelinePutResponse.
I'm not sure it is still going to make sense in TS next gen. Or at least, we may rethink of
the meaningful error code.

> [Storage abstraction] Create backing storage write interface for ATS writers
> ----------------------------------------------------------------------------
>                 Key: YARN-3031
>                 URL: https://issues.apache.org/jira/browse/YARN-3031
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>         Attachments: Sequence_diagram_write_interaction.2.png, Sequence_diagram_write_interaction.png,
YARN-3031.01.patch, YARN-3031.02.patch
> Per design in YARN-2928, come up with the interface for the ATS writer to write to various
backing storages. The interface should be created to capture the right level of abstractions
so that it will enable all backing storage implementations to implement it efficiently.

This message was sent by Atlassian JIRA

View raw message