hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (YARN-7677) Docker image cannot set HADOOP_CONF_DIR
Date Mon, 19 Feb 2018 03:43:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wangda Tan reopened YARN-7677:
------------------------------

Thanks [~rohithsharma] for reporting this issue. 

This is reproducible and resolved after revert of YARN-7677. 

I found launch_container.sh includes following entry:

{code}
export CLASSPATH="$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*:job.jar/*:job.jar/classes/:job.jar/lib/*:$PWD/*"
export APP_SUBMIT_TIME_ENV="1519011470691"
export LD_LIBRARY_PATH="$PWD:$HADOOP_COMMON_HOME/lib/native"
export HADOOP_MAPRED_HOME=""/Users/wtan/project/github/hadoop-common-trunk/hadoop-dist/target/hadoop-3.2.0-SNAPSHOT/""
{code} 

I think {{HADOOP_MAPRED_HOME}} should occur before {{CLASSPATH}}.

cc: [~Jim_Brennan]/[~jlowe]/[~ebadger]

> Docker image cannot set HADOOP_CONF_DIR
> ---------------------------------------
>
>                 Key: YARN-7677
>                 URL: https://issues.apache.org/jira/browse/YARN-7677
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Eric Badger
>            Assignee: Jim Brennan
>            Priority: Major
>             Fix For: 3.1.0
>
>         Attachments: YARN-7677.001.patch, YARN-7677.002.patch, YARN-7677.003.patch, YARN-7677.004.patch,
YARN-7677.005.patch
>
>
> Currently, {{HADOOP_CONF_DIR}} is being put into the task environment whether it's set
by the user or not. It completely bypasses the whitelist and so there is no way for a task
to not have {{HADOOP_CONF_DIR}} set. This causes problems in the Docker use case where Docker
containers will set up their own environment and have their own {{HADOOP_CONF_DIR}} preset
in the image itself. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message