hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers
Date Fri, 12 Jun 2015 21:53:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584148#comment-14584148

Zhijie Shen commented on YARN-3051:

bq. APIs' for querying individual entity/flow/flow run/user and APIs' for querying a set of
entities/flow runs/flows/users. APIs' such a set of flows/users will contain aggregated data.
The reason for separate endpoints for entities, flows, users,etc. is because of the different
tables in HBase/Phoenix schema.

I think we don't store the first class citizen entity in a different way and in different
tables (Li/Vrushali, correct me If I'm wrong). When fetching an entity, it doesn't matter
it is a customized entity or a predefined entity such as ApplicationEntity.

In fact, we have two level of interfaces. One is the storage interface and the other is user-oriented
interface. I think it's a good idea to let the user-oriented interface to have more specific/advanced
APIs to handle the special entity objects, the storage interface could have fewer, more uniformed
APIs to reuse the common logic as much as possible. Thoughts?

bq. Every query param will be received as a String, even timestamp. Now from backing storage
implementation viewpoint, would it make more sense to let these query params be passed as
strings or do datatype conversion ?

I think we need to take the generic type as the param. If it's transformed to a string, it
is likely to be difficult to recover the original type information. For example, when we see
a string "true", how do we know whether it used to be a "true" string too or a true boolean.
Also, "1234567" is a number or is a string that represents a vehicle license.

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051-YARN-2928.003.patch, YARN-3051-YARN-2928.03.patch, YARN-3051-YARN-2928.04.patch,
YARN-3051.wip.02.YARN-2928.patch, YARN-3051.wip.patch, YARN-3051_temp.patch
> Per design in YARN-2928, create backing storage read interface that can be implemented
by multiple backing storage implementations.

This message was sent by Atlassian JIRA

View raw message