hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peeyush Bishnoi <peeyu...@yahoo-inc.com>
Subject Re: Errors running a Map task + third party libraries + .properties
Date Fri, 28 Mar 2008 11:27:37 GMT
Hello ,

I think in your code you have to specify the DistributedCache option to
distribute your properties/configuration file to the remote machines so
that application should load properties/configuration file while your
task is executing. For more information and how to use DistributedCache
please look into the  following URL :

http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#DistributedCache 


Thanks

---
Peeyush


On Fri, 2008-03-28 at 11:49 +0100, Alfonso Olias Sanz wrote:

> Hi there
> 
> I have a Map/Reduce task that loads a .properties file.  In the
> generated jar file I have a
> conf directory with this file.  So the pat look like
> conf/myfile.properties.
> 
> Running the Map/Reduce task inside eclipse (using the eclipse plugin),
> works fine. But when I use the command hadoop jar  MyJar.jar
> inputFolder  outputFolder
> 
> I always get this exception:
> 
> task_200803271613_0028_m_000004_2:
> gaia.cu1.tools.exception.GaiaConfigurationException: Cannot load
> configuration from file conf/idt.properties
> task_200803271613_0028_m_000004_2:      at
> gaia.cu1.tools.util.props.PropertyLoader.load(PropertyLoader.java:163)
> task_200803271613_0028_m_000004_2:      at
> gaia.cu3.idt.infraimpl.Launcher.loadProperties(Launcher.java:95)
> task_200803271613_0028_m_000004_2:      at
> gaia.cu3.idtmapreduce.IdtMapper.setUp(IdtMapper.java:195)
> task_200803271613_0028_m_000004_2:      at
> gaia.cu3.idtmapreduce.IdtMapper.map(IdtMapper.java:228)
> task_200803271613_0028_m_000004_2:      at
> gaia.cu3.idtmapreduce.IdtMapper.map(IdtMapper.java:1)
> task_200803271613_0028_m_000004_2:      at
> org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> task_200803271613_0028_m_000004_2:      at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:208)
> task_200803271613_0028_m_000004_2:      at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2071)
> task_200803271613_0028_m_000004_2: Caused by:
> org.apache.commons.configuration.ConfigurationException: Cannot locate
> configuration source idt.properties
> 
> Any idea???  The jar file also contains third party jar files and I
> included in the manifest the classpath to the relative lib folder, and
> it is working. But not for the properties files.
> Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message