hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Configuration object not loading parameters in unit tests
Date Wed, 16 Jan 2013 18:30:31 GMT
The "known" config files (various *-site.xml) are loaded if they are
discoverable as system resources, i.e. they are present in a directory on
the classpath or in the root of a jar that is on the classpath.

You can perhaps add it to a directory under your root test folder that your
build system adds to the classpath. For instance, in Apache Hadoop which
uses Maven, we add it to the test/resources/ directory of a module and it
is made available in the classpath for test cases automatically.

On Wed, Jan 16, 2013 at 11:35 PM, Jay Vyas <jayunit100@gmail.com> wrote:

> Hi guys:
> I'm trying load some cluster parameters in a unit test, which uses a
> custom filesystem.
> However, my configuration object doesnt seem to be loading:
> For example: conf.get("customparameter") (where "customparameter" is in
> conf/core-site.xml) is returning null.
> To investigate: I tried to print the Configuration object.  Oddly, it
> showed 4 paths, even though none of these files exist locally:
> Configuration : Configuration: core-default.xml, core-site.xml,
> mapred-default.xml, mapred-site.xml, conf/core-site.xml
> How can I force my Configuration object to preferentially load
> conf/core-site.xml  ?  I've added it using
> Configuration.addDefaultResource(...), but it seems that the parameters are
> not getting loaded into the runtime Configuration object.
> Ultimately, the bug I have is that the default filesystem is being
> created, rather than my custom filesystem.
> File system: org.apache.hadoop.fs.LocalFileSystem@4ce2cb55
> --
> Jay Vyas
> http://jayunit100.blogspot.com

Harsh J

View raw message