flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aljoscha Krettek (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-7396) Don't put multiple directories in HADOOP_CONF_DIR in config.sh
Date Wed, 09 Aug 2017 09:53:00 GMT
Aljoscha Krettek created FLINK-7396:

             Summary: Don't put multiple directories in HADOOP_CONF_DIR in config.sh
                 Key: FLINK-7396
                 URL: https://issues.apache.org/jira/browse/FLINK-7396
             Project: Flink
          Issue Type: Bug
          Components: Startup Shell Scripts
    Affects Versions: 1.3.2, 1.4.0
            Reporter: Aljoscha Krettek
             Fix For: 1.4.0, 1.3.3

In config.sh we do this:
# Check if deprecated HADOOP_HOME is set.
if [ -n "$HADOOP_HOME" ]; then
    # HADOOP_HOME is set. Check if its a Hadoop 1.x or 2.x HADOOP_HOME path
    if [ -d "$HADOOP_HOME/conf" ]; then
        # its a Hadoop 1.x
    if [ -d "$HADOOP_HOME/etc/hadoop" ]; then
        # Its Hadoop 2.2+
while our {{HadoopFileSystem}} actually only treats this paths as a single path, not a colon-separated
path: https://github.com/apache/flink/blob/854b05376a459a6197e41e141bb28a9befe481ad/flink-runtime/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopFileSystem.java#L236

I also think that other tools don't assume multiple paths in there and at least one user ran
into the problem on their setup.

This message was sent by Atlassian JIRA

View raw message