hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3051) [Storage abstraction] Create backing storage read interface for ATS readers
Date Wed, 01 Jul 2015 23:14:07 GMT

    [ https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611156#comment-14611156
] 

Zhijie Shen commented on YARN-3051:
-----------------------------------

[~varun_saxena], thanks for being patient about the comments. Here're some more about about
the new patch.

1. We should compare the objects directly instead of converting them to String first.
{code}
136	  private static boolean matchFilter(Object infoValue, Object filterValue) {
137	    // Convert to String and check for now.
138	    return infoValue.toString().equals(filterValue.toString());
139	  }
{code}

2. No one is writing the mapping into APP_FLOW_MAPPING_FILE in the current code base? Are
you suggesting treating it as a property file? What's the rationale? How about using CSV format:
1) searching for user/flowId/flowRunId separately 2) being neutral about path separator.
{code}
59	    prop.setProperty("app1", "user1/flow1/1");
{code}

3. Can we prevent introducing the test oriented configurations into YarnConfiguration?

4. We can do some optimization for the file implementation, such putting created time and
modified time into file name to quickly filter these files without reading them, merging the
entities and overwriting the file to prevent merging again for each query. But that's not
critical here, we can do it later.

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051-YARN-2928.003.patch, YARN-3051-YARN-2928.03.patch, YARN-3051-YARN-2928.04.patch,
YARN-3051-YARN-2928.05.patch, YARN-3051-YARN-2928.06.patch, YARN-3051.Reader_API.patch, YARN-3051.Reader_API_1.patch,
YARN-3051.Reader_API_2.patch, YARN-3051.Reader_API_3.patch, YARN-3051.Reader_API_4.patch,
YARN-3051.wip.02.YARN-2928.patch, YARN-3051.wip.patch, YARN-3051_temp.patch
>
>
> Per design in YARN-2928, create backing storage read interface that can be implemented
by multiple backing storage implementations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message