hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Collier <nick.coll...@verizon.net>
Subject DistributedCache Question
Date Mon, 09 Apr 2012 17:38:08 GMT
Hi,

Using Hadoop 1.0.1 I'm trying to use the DistributedCache to add additional jars to the classpath
used by my Mappers but I can't get it to work. In the run(String[] args) method of my Tool
implementation, I've tried:

FileSystem fs = DistributedFileSystem.get(conf);
DistributedCache.addArchiveToClassPath(new Path("hdfs://localhost:9000/my.app/batch.jar"),
conf, fs);

and

FileSystem fs = DistributedFileSystem.get(conf);
DistributedCache.addArchiveToClassPath(new Path("/my.app/batch.jar"), conf, fs);

and using a local path,

DistributedCache.addArchiveToClassPath(new Path("./lib/batch.jar"), conf);

I've also tried using the addFileToClassPath alternative but also without success. I'm running
in pseudo-distributed mode. Adding the additional jars with -libjars does work, but I would
like to get this working as well as it will be easier to maintain. 

Any help is much appreciated,

Nick




Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message