hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers
Date Fri, 12 Jun 2015 18:26:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14583839#comment-14583839

Li Lu commented on YARN-3051:

Hi [~varun_saxena], thanks for the update! Some of my quick thoughts for discussion...
# I just realized in this JIRA we are creating "backing storage read interface for ATS readers",
but not the user facing ATS reader APIs. I believe these two topics are different: in this
JIRA we're "wiring up" the storage systems, but in ATS reader APIs, we need to deal with user
requirements. This said, I think the main design goal here is to provide a small set of generic
interfaces so that we can easily connect them to our writers. We may want to have some brief
ideas of the potential user facing features (as [~zjshen] mentioned in a previous comment),
but I'm not sure if we need to implement them before we make a concrete design for the storage
read interface. 
# If my understanding in point 1 is right, then perhaps we do not need to quite worry about
the huge list of nulls. Of course, on code level we may want to to some cosmetic fixes, but
since those interfaces are not user facing, making them more general may be more important
I think?
# I still think when doing the v2 interface design, it is fine, if not even beneficial, to
start from scratch rather than thinking about the existing v1 design. If we're not implementing
some v1 features as first-class in v2 storage implementations, maybe we can simply leave them
out from the interfaces to storage level? (I assume we'll have an intermediate layer to do
the wire up between our user facing reader APIs and the storage interfaces. )
# bq. Now from backing storage implementation viewpoint, would it make more sense to let these
query params be passed as strings or do datatype conversion ?
I've got no strong preference on this. Leaving them as a generic type (like string) gives
the storage layer more freedom to interpret the data, but the readers need to ensure they
understand the types by themselves. 

BTW, could you please briefly skim through the list of Jenkins warnings and see if they're
critical? Thanks! 

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051-YARN-2928.003.patch, YARN-3051-YARN-2928.03.patch, YARN-3051-YARN-2928.04.patch,
YARN-3051.wip.02.YARN-2928.patch, YARN-3051.wip.patch, YARN-3051_temp.patch
> Per design in YARN-2928, create backing storage read interface that can be implemented
by multiple backing storage implementations.

This message was sent by Atlassian JIRA

View raw message