hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Iwona Bialynicka-Birula <iwona...@atigeo.com>
Subject Problem debugging MapReduce job under Windows
Date Sat, 07 May 2011 20:56:22 GMT
Hello,

I am trying to debug a Hadoop MapReduce job under Eclipse in Windows and I am running into
a problem when the Hadoop framework tries to set up the staging directory (see the stack trace
below) and change its permissions to 700. It is doing it in RawLocalFileSystem.setPermissions
by calling java.io.File.setReadable(false, false), but under Windows this always fails, since
you're not allowed to set readable to false. Is there a workaround for this to enable debugging
in a Windows environment, or is such a scenario simply not possible?

(I am using Windows 7, Hadoop 0.20.2, Java 6. I have Cygwin installed, but in this case it
is not using it, since it is making the system call directly.)

Thanks,
Iwona

java.io.IOException: Failed to set permissions of path: file:/tmp/hadoop-iwonabb/mapred/staging/iwonabb-1931875024/.staging
to 0700
                at org.apache.hadoop.fs.RawLocalFileSystem.checkReturnValue(RawLocalFileSystem.java:526)
                at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:500)
                at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:310)
                at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
                at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
                at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:799)
                at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
                at java.security.AccessController.doPrivileged(Native Method)
                at javax.security.auth.Subject.doAs(Unknown Source)
                at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
                at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:793)
                at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
                at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:495)
                at com.xpatterns.xrelevance.hadoop.training.crn.CRNTraining.run(CRNTraining.java:168)
                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
                at com.xpatterns.xrelevance.hadoop.training.crn.CRNTraining.main(CRNTraining.java:46)
                at com.xpatterns.xrelevance.tests.CRNTrainingMRvsSingle.compare(CRNTrainingMRvsSingle.java:88)
                at com.xpatterns.xrelevance.tests.CRNTrainingMRvsSingle.compare(CRNTrainingMRvsSingle.java:136)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
                at java.lang.reflect.Method.invoke(Unknown Source)
                at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
                at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
                at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
                at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
                at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
                at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
                at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
                at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
                at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
                at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
                at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
                at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
                at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
                at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
                at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
                at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
                at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
                at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message