hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers
Date Wed, 29 Apr 2015 20:53:07 GMT

    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520214#comment-14520214

Sangjin Lee commented on YARN-3051:

My major concern about this proposal is compatibility. Previously in v1, timeline entity is
globally unique, such that when fetching a single entity before, users only need to provide
<entity type, entity id>. <app id, entity type, entity id> is required to locate
one entity, and theoretically <null, entity type, entity id> will refer to multiple
entities. It probably makes difficult to be compatible to existing use cases.

To hash out that point, existing use cases which previously assumed that entity id was globally
unique would continue to generate entity id's that are globally unique, right? Since existing
use cases (w/o modification) would stick to globally unique entity id's in practice, redefining
the uniqueness requirement to be in the scope of application should not impact existing use
cases. Entity id's that are generated to be unique globally would trivially be unique within
the application scope. The point here is that since this is in the direction of relaxing uniqueness,
stricter use cases (existing use cases) should not be impacted. Let me know your thoughts.

IMO, stating that the entity id's are unique within the scope of applications is not an invitation
for frameworks to generate tons of redundant entity id's. Frameworks (MR, tez, ...) would
likely continue to generate entity id's that are practically unique globally anyway. But the
part of the timeline service, we don't have to have checks for enforcing global uniqueness.

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051.wip.patch, YARN-3051_temp.patch
> Per design in YARN-2928, create backing storage read interface that can be implemented
by multiple backing storage implementations.

This message was sent by Atlassian JIRA

View raw message