hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abdul Qadeer" <qadeer.qad...@gmail.com>
Subject TestMultipleCachefiles with NLineInputFormat
Date Sun, 14 Sep 2008 13:36:47 GMT
If I add "-inputformat", "org.apache.hadoop.mapred.lib.NLineInputFormat",
as one of the job config option in
org.apache.hadoop.streaming.TestMultipleCachefiles
the test case fails saying that it can not find the cached files (Few
relevant lines of the
info messages are pasted in the following).  This test case
works fine with TextInputFormat.  I wonder what goes wrong if I instead use
the NLineInputFormat?

2008-09-14 06:14:49,179 INFO  mapred.TaskTracker
(TaskTracker.java:tryToGetOutputSize(1323)) -
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_200809140613_0001/attempt_200809140613_0001_m_000000_0/output/file.out
in any of the configured local directories
2008-09-14 06:14:49,182 INFO  mapred.TaskTracker
(TaskTracker.java:tryToGetOutputSize(1323)) -
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_200809140613_0001/attempt_200809140613_0001_r_000000_0/output/file.out
in any of the configured local directories
2008-09-14 06:14:49,185 INFO  mapred.TaskInProgress
(TaskInProgress.java:updateStatus(392)) - Error from
attempt_200809140613_0001_m_000000_0: java.lang.RuntimeException:
PipeMapRed.waitOutputThreads(): subprocess failed with code 123
    at
org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:295)
    at
org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:519)
    at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:136)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:56)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:254)
    at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2217)



Abdul Qadeer

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message