hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13070) classloading isolation improvements for cleaner and stricter dependencies
Date Tue, 14 Jun 2016 23:46:30 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sangjin Lee updated HADOOP-13070:
    Attachment: lib.jar

Posted a POC patch that illustrates the idea on how we might prevent a user caller class from
loading classes from the parent classpath. This is admittedly a very preliminary version.

I also attached a quick test driver to test out the idea. You can save them to a directory
structure like
- driver/TestDriver.java
- lib.jar
- user/Test.java

{{Test.java}} tries to load {{foo.Foo}} (in {{lib.jar}}) via various ways of classloading
(you can uncomment and comment those approaches in {{Test.java}} to test each). You can place
{{lib.jar}} either in the main classpath ("parent" classpath) or as an argument to {{TestDriver}}
("user" classpath) and see what happens. For example,

java -cp /Users/alice/git/hadoop-trunk/hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-alpha1-SNAPSHOT.jar:/Users/alice/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/alice/.m2/repository/org/slf4j/slf4j-api/1.7.7/slf4j-api-1.7.7.jar:.
TestDriver ../user:../lib.jar


java -cp /Users/alice/git/hadoop-trunk/hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-alpha1-SNAPSHOT.jar:/Users/alice/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/alice/.m2/repository/org/slf4j/slf4j-api/1.7.7/slf4j-api-1.7.7.jar:.:../lib.jar
TestDriver ../user

> classloading isolation improvements for cleaner and stricter dependencies
> -------------------------------------------------------------------------
>                 Key: HADOOP-13070
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13070
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>            Priority: Critical
>         Attachments: HADOOP-13070.poc.01.patch, Test.java, TestDriver.java, classloading-improvements-ideas-v.3.pdf,
classloading-improvements-ideas.pdf, classloading-improvements-ideas.v.2.pdf, lib.jar
> Related to HADOOP-11656, we would like to make a number of improvements in terms of classloading
isolation so that user-code can run safely without worrying about dependency collisions with
the Hadoop dependencies.
> By the same token, it should raised the quality of the user code and its specified classpath
so that users get clear signals if they specify incorrect classpaths.
> This will contain a proposal that will include several improvements some of which may
not be backward compatible. As such, it should be targeted to the next major revision of Hadoop.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message