hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: core-site.xml file is being ignored by new Configuration()
Date Tue, 29 Jan 2013 01:58:54 GMT
Can you try prepending the path with file:// or pass in a URL and use the method which takes
in a URL?

I remember class loading issues with paths without the file: scheme info from a different
scenario (while playing with logj4 config files).

HTH,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/

On Jan 16, 2013, at 4:33 PM, Jay Vyas wrote:

> Hi guys: 
> 
> I've finally extracted my problem of loading a special filesystem
> into a unit test.
> 
> Below, clearly, Im creating a raw configuration and adding a single resource to it (core-site.xml).
> 
> Afterwards, i print to confirm that the file exists.  
> 
> Finally, I try to create a FileSystem from that file.  However, no file system is created
:(.
> 
> Any thoughts on why this core-site.xml file is being ignoreed?
> To note: I've also tried putting core-site.xml in the working directory, and in a local
conf/ directory.  This absolute path was a last effort to really confirm that the local path
wasn't somehow being mangled by the junit runtime.
> 
> @Test
>     public void test() throws Exception{
>         Configuration conf = new Configuration(false);
>         System.out.println("b4:"+conf);
>         String theXMLFile="/Users/jayunit100/Development/blah/conf/core-site.xml";
>         conf.addResource(theXMLFile);
>         System.out.println("File exists : " + theXMLFile.length());
>         System.out.println("after:"+conf);
>         System.out.println("file sytem " + conf.get("fs.default.name"));
>         conf.writeXml(System.out);
> 
>         
>         System.out.println("Working directory = " + System.getProperty("user.dir"));
>         org.apache.hadoop.fs.FileSystem fs = org.apache.hadoop.fs.FileSystem.get(conf);
>         System.out.println("before test filesystem = " + fs);
>         //WordCount.main(new String[]{"resources/wordcount.txt","wordcountout.txt"});
>     }
> 
> -- 
> Jay Vyas
> http://jayunit100.blogspot.com


Mime
View raw message