hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject local job directory customization
Date Mon, 13 Sep 2010 22:43:36 GMT
Hi,
I saw the following in our unit test log on build machine:

Caused by: org.apache.hadoop.util.Shell$ExitCodeException: chmod:
cannot access `/tmp/hadoop-build/mapred/system/job_local_0002/job.split':
No such file or directory

	at org.apache.hadoop.util.Shell.runCommand(Shell.java:195)
	at org.apache.hadoop.util.Shell.run(Shell.java:134)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
	at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
	at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
	at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:492)
	at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:484)
	at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:286)
	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:373)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:526)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:507)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:414)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:406)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:285)
	at org.apache.hadoop.mapred.JobClient.writeSplitsFileHeader(JobClient.java:1057)
	at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:873)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:822)
	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:771)
	at org.apache.hadoop.mapred.HadoopClient.runJob(HadoopClient.java:177)

I guess the reason was that multiple unit tests (from different build
projects) interfere with each other.
Is there a way to customize local job directory ?

Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message