hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kavali, Devaraj" <devaraj.kav...@intel.com>
Subject RE: Running MRV1 code on YARN
Date Tue, 08 Apr 2014 12:42:32 GMT
As per the given exception stack trace, it is trying to use local file system. Can you check
whether you have configured the file system configurations with HDFS?

Thanks
Devaraj K

From: divye sheth [mailto:divs.sheth@gmail.com]
Sent: Tuesday, April 08, 2014 5:37 PM
To: user@hadoop.apache.org
Subject: Running MRV1 code on YARN

Hi,

I have installed Hadoop 2.2.0 along with YARN and am trying to submit a MRV1 job already written
to YARN.

The job does not even submit and it prints the following stack trace on console:

2014-04-08 16:56:11 UserGroupInformation [ERROR] PriviledgedActionException as:eureka (auth:SIMPLE)
cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access `/user/eureka54695942/.staging/job_local54695942_0001':
No such file or directory

Exception in thread "main" org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access
`/user/eureka54695942/.staging/job_local54695942_0001': No such file or directory

        at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
        at org.apache.hadoop.util.Shell.run(Shell.java:188)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
        at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
        at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
        at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:593)
        at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:584)
        at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:427)
        at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:579)
        at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:786)
        at org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:746)
        at org.apache.hadoop.mapred.JobClient.access$400(JobClient.java:177)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:963)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:948)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:948)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)

My question here is if you notice the staging location which it is trying to clean I do not
have any such user in the /user directory in hdfs. It somehow appends the jobId to the username
and creates staging area there. Any reason for this? Please let me know what am I doing wrong.
How can I make sure it goes to the user that I have created i.e. eureka and not eureka$JOBID.

I am using CDH4.

Thanks
Divye Sheth

Mime
View raw message