hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Itzhak Pan <itzhak....@gmail.com>
Subject Re: RE: Re:Problem debugging MapReduce job under Windows
Date Mon, 09 May 2011 01:12:00 GMT
I encountered the same problem. It is because the app fails to call 'chmod'
in windows. My solution is to include cygwin directory in PATH.

在 2011 5 8 11:29,"Iwona Bialynicka-Birula" <iwona.bb@atigeo.com>写道:
> Thanks for the reply, but the command you suggest refers to the Hadoop
file system and I am having trouble with the local file system - in
particular a subset of hadoop.tmp.dir. Also, even if I grant full
permissions to this folder initially, Hadoop still tries to set them to 700,
which is not allowed under Windows.
>
> Thanks,
> Iwona
>
> -----Original Message-----
> From: hadoopfan [mailto:shhaijava@126.com]
> Sent: Saturday, May 07, 2011 5:16 PM
> To: common-user@hadoop.apache.org
> Subject: Re:Problem debugging MapReduce job under Windows
>
> hadoop dfs -chmod 777 /tmp
>
>
>
> At 2011-05-08 04:56:22,"Iwona Bialynicka-Birula" <iwona.bb@atigeo.com>
wrote:
>
>>Hello,
>>
>>I am trying to debug a Hadoop MapReduce job under Eclipse in Windows and I
am running into a problem when the Hadoop framework tries to set up the
staging directory (see the stack trace below) and change its permissions to
700. It is doing it in RawLocalFileSystem.setPermissions by calling
java.io.File.setReadable(false, false), but under Windows this always fails,
since you're not allowed to set readable to false. Is there a workaround for
this to enable debugging in a Windows environment, or is such a scenario
simply not possible?
>>
>>(I am using Windows 7, Hadoop 0.20.2, Java 6. I have Cygwin installed, but
in this case it is not using it, since it is making the system call
directly.)
>>
>>Thanks,
>>Iwona
>>
>>java.io.IOException: Failed to set permissions of path:
file:/tmp/hadoop-iwonabb/mapred/staging/iwonabb-1931875024/.staging to 0700
>> at
org.apache.hadoop.fs.RawLocalFileSystem.checkReturnValue(RawLocalFileSystem.java:526)
>> at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:500)
>> at
org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:310)
>> at
org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
>> at
org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:799)
>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Unknown Source)
>> at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>> at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:793)
>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:465)
>> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:495)
>> at
com.xpatterns.xrelevance.hadoop.training.crn.CRNTraining.run(CRNTraining.java:168)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at
com.xpatterns.xrelevance.hadoop.training.crn.CRNTraining.main(CRNTraining.java:46)
>> at
com.xpatterns.xrelevance.tests.CRNTrainingMRvsSingle.compare(CRNTrainingMRvsSingle.java:88)
>> at
com.xpatterns.xrelevance.tests.CRNTrainingMRvsSingle.compare(CRNTrainingMRvsSingle.java:136)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>> at java.lang.reflect.Method.invoke(Unknown Source)
>> at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>> at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>> at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>> at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>> at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
>> at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
>> at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>> at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
>> at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
>> at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
>> at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message