hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuanyuan Tian <yt...@us.ibm.com>
Subject files are inaccessible after HDFS upgrade from 0.18.1 to 1.19.0
Date Tue, 27 Jan 2009 01:18:00 GMT


I just upgraded hadoop from 0.18.1 to 0.19.0 following the instructions on
http://wiki.apache.org/hadoop/Hadoop_Upgrade. After upgrade, I run fsck,
everything seems fine. All the files can be listed in hdfs and the sizes
are also correct. But when a mapreduce job tries to read the files as
input, the following error messages are returned for some of the files:

java.io.IOException: Could not obtain block: blk_-2827537120880440835_1131
             at org.apache.hadoop.hdfs.DFSClient
             at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo
             at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read
             at java.io.DataInputStream.read(DataInputStream.java:150)
             at java.io.ObjectInputStream$PeekInputStream.read
             at java.io.ObjectInputStream$PeekInputStream.readFully
             at java.io.ObjectInputStream$BlockDataInputStream.readShort
             at java.io.ObjectInputStream.readStreamHeader
             at java.io.ObjectInputStream.(ObjectInputStream.java:298)

             at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
             at org.apache.hadoop.mapred.Child.main(Child.java:155)

I also tried to browse these files through the HDFS web interface,
java.io.EOFException is returned.

Is there any way to recover the files?

Thanks very much,

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message