hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers
Date Mon, 30 Mar 2015 20:35:53 GMT

    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387351#comment-14387351
] 

Zhijie Shen commented on YARN-3051:
-----------------------------------

bq. Wanted to know, how will app context information be used by storage implementation ? I
was initially thinking it will be passed as info inside the entity.

bq. I think for reader APIs,we need to support use cases both with and without the context
info.

I'd to say it's more than a context info issue. It's actually how we identify one entity.
In YTS v1, we use (entity type, entity id) to globally identify an unique timeline entity.
In YTS v2, according to the data schema discussion, if my understanding is correct, we want
to use (cluster id, user id, flow name, flow version*, flow run, app id, entity type, entity
id) to globally identify the entity. It means that previously, there could just be one entity("My_Type",
"My_ID"), but now we can have such a entity for App1 and the same one for App2 if App1 !=
App2.

GetEntity(entity type, entity value) won't be able to locate a single entity, but possibly
multiple ones, unless we say the data schema PK is for storage performance, and we still uniquely
identify a entity by (entity type, entity id) across app, flow, user and cluster.

bq. I remember we had some discussion on RESTful calls we need to support (and some documents
drafted by Vrushali C), but I couldn't find it...

I hope the existing queries can continue work on YTS v2 to some extent, even though there
will be incompatible changes. For example, the aforementioned identifier semantics change
will significantly change the API to get a single entity.

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>
> Per design in YARN-2928, create backing storage read interface that can be implemented
by multiple backing storage implementations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message