hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7190) Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath
Date Thu, 14 Sep 2017 14:20:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166359#comment-16166359

Jason Lowe commented on YARN-7190:

Separate directory for these that's only pulled into the classpath when necessary would be
my preference.  That's already done for the stuff in share/hadoop/tools, so we could have
something like share/hadoop/yarn/server/ or share/hadoop/yarn/nodemanager/ or whatever where
jars that are specific to YARN servers and/or the YARN nodemanager can be placed.  

Shading has a lot of pitfalls and to me is more of a last resort.  Shading is for solving
the problem where the client _does_ need the jars but they will conflict with other jars that
the user provides.  In this case it's not necessary since the client doesn't need these jars
at all.  

> Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath
> ------------------------------------------------------------------------------------
>                 Key: YARN-7190
>                 URL: https://issues.apache.org/jira/browse/YARN-7190
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineclient, timelinereader, timelineserver
>            Reporter: Vrushali C
> [~jlowe] had a good observation about the user classpath getting extra jars in hadoop
2.x brought in with TSv2.  If users start picking up Hadoop 2,x's version of HBase jars instead
of the ones they shipped with their job, it could be a problem.
> So when TSv2 is to be used in 2,x, the hbase related jars should come into only the NM
classpath not the user classpath.
> Here is a list of some jars
> {code}
> commons-csv-1.0.jar
> commons-el-1.0.jar
> commons-httpclient-3.1.jar
> disruptor-3.3.0.jar
> findbugs-annotations-1.3.9-1.jar
> hbase-annotations-1.2.6.jar
> hbase-client-1.2.6.jar
> hbase-common-1.2.6.jar
> hbase-hadoop2-compat-1.2.6.jar
> hbase-hadoop-compat-1.2.6.jar
> hbase-prefix-tree-1.2.6.jar
> hbase-procedure-1.2.6.jar
> hbase-protocol-1.2.6.jar
> hbase-server-1.2.6.jar
> htrace-core-3.1.0-incubating.jar
> jamon-runtime-2.4.1.jar
> jasper-compiler-5.5.23.jar
> jasper-runtime-5.5.23.jar
> jcodings-1.0.8.jar
> joni-2.1.2.jar
> jsp-2.1-6.1.14.jar
> jsp-api-2.1-6.1.14.jar
> jsr311-api-1.1.1.jar
> metrics-core-2.2.0.jar
> servlet-api-2.5-6.1.14.jar
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message