hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5189) Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl
Date Wed, 01 Jun 2016 19:17:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-5189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310937#comment-15310937
] 

Sangjin Lee commented on YARN-5189:
-----------------------------------

Thanks for the patch [~jrottinghuis]! These are some early high level comments.

(1) The distributedshell project {{pom.xml}} should add the {{<type>test-jar</type>}}
to the timelineservice dependency; the compilation breaks without that change.

(2) I believe it is the hadoop convention that all unit test class names start with {{Test}}.
I'm not sure where that is specified, but I've seen that pretty much universally throughout
the hadoop code. Since the file-based reader/writer impl classes are not really unit tests
in and of themselves, I'm not sure if we want to rename them to {{Test*}}. I think it is sufficient
to move them to {{src/test}} without renaming and make it clear that they are solely for unit
test purposes. Can we do that instead? That might also have a side benefit of reducing the
amount of changes here.

(3) I think the key point of this JIRA is to change the default values in the configuration.
Currently the reader/write impl values are set to the file-based ones. As part of this JIRA,
we should change them to the hbase-based ones (in {{YarnConfiguration.java}} as well as {{yarn-default.xml}}).
It also means that we need to look through the existing code carefully and make sure we override
these configs when we really need the file-based ones.

(4) I believe Joep was able to reproduce but it appears that some {{timelineservice}} unit
tests are taking much longer and are having issues. We need to see why that is. I suspect
it is related with (3).

> Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl
> ------------------------------------------------------------------------------
>
>                 Key: YARN-5189
>                 URL: https://issues.apache.org/jira/browse/YARN-5189
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Joep Rottinghuis
>            Assignee: Joep Rottinghuis
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-5189-YARN-2928.01.patch
>
>
> [~naganarasimha_gr@apache.org] questioned whether it made sense to default to an implementation
that doesn't support all functionality.
> [~sjlee0] opened YARN-5174 to track updating the documentation for ATS to reflect the
default shifting to the fully functional HBase implementation.
> It makes sense to remove a partial implementation, but on the other hand it is still
handing in testing. Hence this jira to move the file based implementations to the test package
and to make the HBase impls the default.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message