Mailing-List: contact dev-help@reef.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@reef.apache.org
Date: Fri, 8 Dec 2017 05:50:01 +0000 (UTC)
From: "John Yang (JIRA)" <jira@apache.org>
To: dev@reef.apache.org
Message-ID: <JIRA.13117776.1510476259000.415508.1512712201289@Atlassian.JIRA>
In-Reply-To: <JIRA.13117776.1510476259000@Atlassian.JIRA>
References: <JIRA.13117776.1510476259000@Atlassian.JIRA> <JIRA.13117776.1510476259030@jira-lw-us.apache.org>
Subject: [jira] [Commented] (REEF-1959) Fix bug in
 ClassPathBuilder#addAllToSuffix
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 08 Dec 2017 05:50:09 -0000


    [ https://issues.apache.org/jira/browse/REEF-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283091#comment-16283091 ] 

John Yang commented on REEF-1959:
---------------------------------

Resolved by https://github.com/apache/reef/pull/1416

> Fix bug in ClassPathBuilder#addAllToSuffix
> ------------------------------------------
>
>                 Key: REEF-1959
>                 URL: https://issues.apache.org/jira/browse/REEF-1959
>             Project: REEF
>          Issue Type: Bug
>          Components: REEF Runtime YARN
>            Reporter: Geon-Woo Kim
>            Assignee: Geon-Woo Kim
>             Fix For: 0.17
>
>
> The method is written as
> {code:java}
>   void addAllToSuffix(final String... entries) {
>     for (final String classPathEntry : entries) {
>       /** should be add to suffix **/
>       this.addToPrefix(classPathEntry);
>     }
>   }
> {code}
> and I believe it should be
> {code:java}
>   void addAllToSuffix(final String... entries) {
>     for (final String classPathEntry : entries) {
>       this.addToSuffix(classPathEntry);
>     }
>   }
> {code}
> The method is used to add some hadoop dependencies to the class path of the driver jvm, like addAllToSuffix(SOME_HADOOP_DEPENDENCIES). So we expect *-classpath* looks like *APPLICATION_JAR:SOME_HADOOP_DEPENDENCIES*, but the bug makes it as *SOME_HADOOP_DEPENDENCIES:APPLICATION_JAR*, which causes the class loader firstly looks up transitive dependencies from the HADOOP_DEPENDENCIES. We cannot use higher versions of guava or netty which are also defined on those HADOOP_DEPENDENCIES, with the bug. I guess that this bug also causes REEF-1897 .


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)