hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5667) Move HBase backend code in ATS v2 into its separate module
Date Wed, 05 Oct 2016 19:54:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549768#comment-15549768
] 

Vrushali C commented on YARN-5667:
----------------------------------

Thanks [~haibochen], I took a quick look and it seems to be good. It does not look like you
have updated the hbase version in this patch, right? Since you had mentioned you wanted to
build with hbase 2.0. Am I correct in thinking you  tested this with hbase 1.1.3?  

Also, there are a lot of files moved around, so wanted to confirm, the classes being moved
are the ones primarily under  hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/.
If possible, could you share a listing of what remains in the  hadoop-yarn-server-timelineservice/src/main?


Also, did you enable the coprocessor on your pseudo cluster during your checks? Did you recollect
the counters aggregated to the flow level? 

> Move HBase backend code in ATS v2  into its separate module
> -----------------------------------------------------------
>
>                 Key: YARN-5667
>                 URL: https://issues.apache.org/jira/browse/YARN-5667
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>         Attachments: part1.yarn5667.prelim.patch, part2.yarn5667.prelim.patch, part3.yarn5667.prelim.patch,
part4.yarn5667.prelim.patch, part5.yarn5667.prelim.patch
>
>
> The HBase backend code currently lives along with the core ATS v2 code in hadoop-yarn-server-timelineservice
module. Because Resource Manager depends on hadoop-yarn-server-timelineservice, an unnecessary
dependency of the RM module on HBase modules is introduced (HBase backend is pluggable, so
we do not need to directly pull in HBase jars). 
> In our internal effort to try ATS v2 with HBase 2.0 which depends on Hadoop 3, we encountered
a circular dependency during our builds between HBase2.0 and Hadoop3 artifacts.
> {code}
> hadoop-mapreduce-client-common, hadoop-yarn-client, hadoop-yarn-server-resourcemanager,
hadoop-yarn-server-timelineservice, hbase-server, hbase-prefix-tree, hbase-hadoop2-compat,
hadoop-mapreduce-client-jobclient, hadoop-mapreduce-client-common]
> {code}
> This jira proposes we move all HBase-backend-related code from hadoop-yarn-server-timelineservice
into its own module (possible name is yarn-server-timelineservice-storage) so that core RM
modules do not depend on HBase modules any more.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message