hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Eagles (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14387) new Configuration().get() fails if core-site.xml isn't on the classpath
Date Mon, 08 May 2017 16:37:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001049#comment-16001049
] 

Jonathan Eagles commented on HADOOP-14387:
------------------------------------------

[~stevel@apache.org], Could use some help reproducing the case you are seeing. Some background.
The original patch tries to maintain the policy that all default resources are optional by
checking the reader. 

{code:title=Configuration#loadResource}
      ...
      if (reader == null) {
        if (quiet) {
          return null;
        }
        throw new RuntimeException(resource + " not found");
      }
      ...
{code}

Where I am having confusion is the stack trace provided doesn't match the description of how
to reproduce. I verified the bug in the stack trace is fixed. I have verified in my setup
that core-site.xml is optional using the below test case. Can you help me understand how to
reproduce the case you are having? 

{code:title=FailConf.java}
import org.apache.hadoop.conf.Configuration;
public class FailConf {
  public static void main(String[] args) {
    Configuration conf = new Configuration();
    System.out.println(conf.get("hadoop.common.configuration.version"));
    System.out.println(conf.toString());
  }
}
{code}

{code:title=bash}
# build hadoop 2.9
$ mvn clean install package -Pdist -Dtar -DskipTests -Dmaven.javadoc.skip
# build FailConf.java
$ javac -cp ./:./hadoop-client/target/hadoop-client-2.9.0-SNAPSHOT/share/hadoop/client/lib/*
FailConf.java
# run FailConf with core-default.xml and no core-site.xml
$ java -cp ./:./hadoop-client/target/hadoop-client-2.9.0-SNAPSHOT/share/hadoop/client/lib/*
FailConf
# Success
# 0.23.0
# Configuration: core-default.xml, core-site.xml
{code}

> new Configuration().get() fails if core-site.xml isn't on the classpath
> -----------------------------------------------------------------------
>
>                 Key: HADOOP-14387
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14387
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: conf
>    Affects Versions: 3.0.0-alpha3
>         Environment: test run in downstream project with no core-site in test/resources
>            Reporter: Steve Loughran
>            Assignee: Jonathan Eagles
>            Priority: Blocker
>         Attachments: HADOOP-14387.1.patch
>
>
> If you try to create a config via {{new Configuration()}} and there isn't a {{core-site.xml}}
on the CP, you get a stack trace. Previously it'd just skip the failure to load.
> This is a regression which breaks downstream apps that don't need a core-site to run,
but do want to load core-default &c



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message