hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9450) HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
Date Wed, 10 Sep 2014 17:17:33 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128753#comment-14128753
] 

Sangjin Lee commented on HADOOP-9450:
-------------------------------------

This was borne out of an investigation of a hadoop issue at our company. It appears that prior
to this the hadoop configuration directory (HADOOP_CONF_DIR) was always the first entry in
the classpath, regardless of HADOOP_USER_CLASSPATH_FIRST or HADOOP_CLASSPATH. But after HADOOP-9450,
if HADOOP_USER_CLASSPATH_FIRST is set and the user provides his/her version of *-site.xml
through HADOOP_CLASSPATH, the user would end up trumping the hadoop configuration. And I believe
it is still the case after Allen's changes (HADOOP-9902).

Is this an intended behavior? What I'm not sure of is whether we expect the client to be able
to override the site.xml files the hadoop configuration provides. If that is true, then it
is working as desired. If not, we'd need to fix this behavior.

Thoughts?


> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of APpended
> --------------------------------------------------------------------------------------
>
>                 Key: HADOOP-9450
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9450
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>            Reporter: Mitch Wyle
>            Assignee: Harsh J
>             Fix For: 1-win, 2.1.0-beta, 1.3.0
>
>         Attachments: HADOOP-9450-branch-1-win.patch, HADOOP-9450-branch-1.patch, HADOOP-9450-branch-2.patch,
HADOOP-9450.patch, HADOOP-9450.patch
>
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS are pre-pended
BEFORE the user's HADOOP_CLASSPATH.  Therefore there is no way to put your own JARs in front
of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order.  Failing that, we would like to add a command
line option to override this behavior and enable a user's JARs to be found before the wrong
ones in the Hadoop library paths.
> We always welcome your opinions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message