hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Usman Waheed <usm...@opera.com>
Subject Hadoop Reduce Job errors, job gets killed.
Date Mon, 06 Apr 2009 14:09:39 GMT
Hi,

My Hadoop Map/Reduce is giving the following error message right about 
when it is 95% complete with the reducing step on one node. The process 
gets killed. The error message from the logs are noted below.
*java.io.IOException: Filesystem closed*, any ideas please?

2009-04-06 10:41:07,202 INFO org.apache.hadoop.streaming.PipeMapRed: 
Records R/W=10263370/642860
2009-04-06 10:41:17,203 INFO org.apache.hadoop.streaming.PipeMapRed: 
Records R/W=10263370/1033247
2009-04-06 10:41:27,437 INFO org.apache.hadoop.streaming.PipeMapRed: 
Records R/W=10263370/1844222
2009-04-06 10:41:37,438 INFO org.apache.hadoop.streaming.PipeMapRed: 
Records R/W=10263370/2884839
2009-04-06 10:41:44,350 WARN org.apache.hadoop.streaming.PipeMapRed: 
java.io.IOException: Filesystem closed
    at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:166)
    at org.apache.hadoop.dfs.DFSClient.access$500(DFSClient.java:58)
    at 
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.writeChunk(DFSClient.java:2104)
    at 
org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:141)
    at 
org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:124)
    at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:112)
    at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86)
    at 
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:41)
    at java.io.DataOutputStream.write(DataOutputStream.java:90)
    at 
org.apache.hadoop.mapred.TextOutputFormat$LineRecordWriter.writeObject(TextOutputFormat.java:72)
    at 
org.apache.hadoop.mapred.TextOutputFormat$LineRecordWriter.write(TextOutputFormat.java:87)
    at org.apache.hadoop.mapred.ReduceTask$2.collect(ReduceTask.java:315)
    at 
org.apache.hadoop.streaming.PipeMapRed$MROutputThread.run(PipeMapRed.java:346)

2009-04-06 10:41:44,478 INFO org.apache.hadoop.streaming.PipeMapRed: 
MRErrorThread done
2009-04-06 10:41:44,478 INFO org.apache.hadoop.streaming.PipeMapRed: 
PipeMapRed.waitOutputThreads(): subprocess failed with code 141 in 
org.apache.hadoop.streaming.PipeMapRed
2009-04-06 10:41:44,480 INFO org.apache.hadoop.streaming.PipeMapRed: 
mapRedFinished
2009-04-06 10:41:44,480 INFO org.apache.hadoop.streaming.PipeMapRed: 
PipeMapRed.waitOutputThreads(): subprocess failed with code 141 in 
org.apache.hadoop.streaming.PipeMapRed
2009-04-06 10:41:44,481 WARN org.apache.hadoop.mapred.TaskTracker: Error 
running child
java.io.IOException: Filesystem closed
    at org.apache.hadoop.dfs.DFSClient.checkOpen(DFSClient.java:166)
    at org.apache.hadoop.dfs.DFSClient.access$500(DFSClient.java:58)
    at 
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.flush(DFSClient.java:2176)
    at java.io.FilterOutputStream.flush(FilterOutputStream.java:123)
    at java.io.DataOutputStream.flush(DataOutputStream.java:106)
    at 
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:66)
    at 
org.apache.hadoop.mapred.TextOutputFormat$LineRecordWriter.close(TextOutputFormat.java:99)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:340)
    at 
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2084)

Mime
View raw message