hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "gziyang" <gziy...@gmail.com>
Subject EOFException causes failure.
Date Sun, 04 Jan 2009 05:39:12 GMT
Hello,

I would like to run a mapreduce program on hadoop 0.19.0. The program is designed to iterate
for 1000 times. However, in a certain iteration, the program fails due to an EOFException,
but the jobs complete for all the previous iterations. Sometimes the program fails in the
5th iteration, while in other times, the program might fail until the 9th iteration. It seems
a little random.

A similar situation has been discussed for hadoop version 0.2, but it is said the bug has
been fixed in version 0.3. Anybody knows that what is wrong with my hadoop 0.19.0?

...
09/01/04 12:54:37 INFO mapred.JobClient:  map 60% reduce 0%
09/01/04 12:54:41 INFO mapred.JobClient:  map 66% reduce 0%
09/01/04 12:54:41 INFO mapred.JobClient: Task Id : attempt_200901041244_0007_m_000011_0, Status
: FAILED
java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at java.io.DataInputStream.readFully(DataInputStream.java:152)
        at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)
        at org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43)
        at org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
        at org.apache.hadoop.mapred.Child.main(Child.java:155)
09/01/04 12:54:43 INFO mapred.JobClient:  map 70% reduce 1%
09/01/04 12:54:45 INFO mapred.JobClient:  map 78% reduce 1%
09/01/04 12:54:46 INFO mapred.JobClient:  map 81% reduce 2%
09/01/04 12:54:48 INFO mapred.JobClient:  map 83% reduce 2%
09/01/04 12:54:48 INFO mapred.JobClient: Task Id : attempt_200901041244_0007_m_000011_1, Status
: FAILED
java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at java.io.DataInputStream.readFully(DataInputStream.java:152)
        at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)
        at org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43)
        at org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
        at org.apache.hadoop.mapred.Child.main(Child.java:155)
09/01/04 12:54:50 INFO mapred.JobClient:  map 83% reduce 3%
09/01/04 12:54:51 INFO mapred.JobClient:  map 86% reduce 4%
09/01/04 12:54:53 INFO mapred.JobClient:  map 86% reduce 6%
09/01/04 12:54:53 INFO mapred.JobClient: Task Id : attempt_200901041244_0007_m_000011_2, Status
: FAILED
java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at java.io.DataInputStream.readFully(DataInputStream.java:152)
        at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)
        at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)
        at org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43)
        at org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
        at org.apache.hadoop.mapred.Child.main(Child.java:155)
09/01/04 12:54:55 INFO mapred.JobClient:  map 86% reduce 9%
09/01/04 12:54:56 INFO mapred.JobClient:  map 88% reduce 10%
09/01/04 12:54:58 INFO mapred.JobClient:  map 88% reduce 15%
java.io.IOException: Job failed!

Thanks a lot!

2009-01-04 



gziyang 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message