hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shuja Rehman <shujamug...@gmail.com>
Subject Distributed Cache not working
Date Mon, 04 Apr 2011 15:02:24 GMT
Hi All,
I have implemented the distributed cache according to following article.

http://chasebradford.wordpress.com/2011/02/05/distributed-cache-static-objects-and-fast-setup/

but when i run the program over the cluster, i am getting the following
exceptions.

SEVERE: null
java.io.FileNotFoundException: File does not exist:
tmp/extract/7ecfcd44-dd47-4dbf-a12e-464e9d285762
        at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:519)
        at
org.apache.hadoop.filecache.DistributedCache.getFileStatus(DistributedCache.java:362)
        at
org.apache.hadoop.filecache.TrackerDistributedCacheManager.determineTimestamps(TrackerDistributedCacheManager.java:750)
        at
org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:706)
        at
org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:609)
        at org.apache.hadoop.mapred.JobClient.access$300(JobClient.java:170)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:808)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
        at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:793)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:495)
        at
AcuteNightProcessor.ProcessorDriver.runAlertJob(ProcessorDriver.java:346)
        at
AcuteNightProcessor.ProcessorDriver.main(ProcessorDriver.java:461)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:186)

Here is the code :

 FileSystem fs = FileSystem.get(config);
       Path temp = new Path("tmp/extract", UUID.randomUUID().toString());
       ObjectOutputStream os = new ObjectOutputStream(fs.create(temp));
       os.writeObject(dcDto);
       os.close();
       fs.deleteOnExit(temp);

       // Register the file in the DC.  Open the local file "targets"
       DistributedCache.addCacheFile(new URI(temp+"#targets"), config);
       DistributedCache.createSymlink(config);


Does anybody know the solution?

Thanks




-- 
Regards
Shuja-ur-Rehman Baig
<http://pk.linkedin.com/in/shujamughal>

Mime
View raw message