hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers
Date Tue, 30 Jun 2015 18:11:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608774#comment-14608774
] 

Zhijie Shen commented on YARN-3051:
-----------------------------------

bq. One more thing I have changed is limit of entities by default have been kept has 100 instead
of 1000. 1000 seemed too many. Thoughts ?

Sure, just noticed that the previous limit is 100 too.

bq. Ok, so the last entity entry should be the one returned ?

It's not that straightforward. For example, I can put entity 1 twice: one contains event 1
and the other contains event 2. In fact, when I want to retrieve the entity 1 with event field
included. I actually want to have both events. I can see two choices: one is to merge the
entity data at the write path and the other at the read path.

bq. Added in TimelineUtils because dumpTimelineRecordtoJSON used by FS Writer was also put
in the same class.

That method is used by downstream project (e.g., tez) to logging/debugging the ATS integration.
And this all getters of the data model objects are annotated. The method is applicable to
all these objects. On the other side, we only annotate "jasonsetter" for TimelineEntity, such
that getTimelineRecordFromJSON is not generalized enough for all purpose, but for FS impl
only now. Maybe we can hold back the method and promote it to public api once we see real
use case of it. Thoughts?

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051-YARN-2928.003.patch, YARN-3051-YARN-2928.03.patch, YARN-3051-YARN-2928.04.patch,
YARN-3051-YARN-2928.05.patch, YARN-3051.Reader_API.patch, YARN-3051.Reader_API_1.patch, YARN-3051.Reader_API_2.patch,
YARN-3051.Reader_API_3.patch, YARN-3051.Reader_API_4.patch, YARN-3051.wip.02.YARN-2928.patch,
YARN-3051.wip.patch, YARN-3051_temp.patch
>
>
> Per design in YARN-2928, create backing storage read interface that can be implemented
by multiple backing storage implementations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message