hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuanyuan Tian <yt...@us.ibm.com>
Subject files are inaccessible after HDFS upgrade from 0.18.1 to 1.19.0
Date Tue, 27 Jan 2009 01:18:00 GMT


Hi,

I just upgraded hadoop from 0.18.1 to 0.19.0 following the instructions on
http://wiki.apache.org/hadoop/Hadoop_Upgrade. After upgrade, I run fsck,
everything seems fine. All the files can be listed in hdfs and the sizes
are also correct. But when a mapreduce job tries to read the files as
input, the following error messages are returned for some of the files:

java.io.IOException: Could not obtain block: blk_-2827537120880440835_1131
file=/user/hmail/NSF/50k_nntp_clean2.nsf.fs.kvp
             at org.apache.hadoop.hdfs.DFSClient
$DFSInputStream.chooseDataNode(DFSClient.java:1708)
             at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo
(DFSClient.java:1536)
             at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read
(DFSClient.java:1663)
             at java.io.DataInputStream.read(DataInputStream.java:150)
             at java.io.ObjectInputStream$PeekInputStream.read
(ObjectInputStream.java:2283)
             at java.io.ObjectInputStream$PeekInputStream.readFully
(ObjectInputStream.java:2296)
             at java.io.ObjectInputStream$BlockDataInputStream.readShort
(ObjectInputStream.java:2767)
             at java.io.ObjectInputStream.readStreamHeader
(ObjectInputStream.java:798)
             at java.io.ObjectInputStream.(ObjectInputStream.java:298)
             at
emailanalytics.importer.parallelimport.EmailContentRecordReader.(EmailContentRecordReader.java:32)

             at
emailanalytics.importer.parallelimport.EmailContentFormat.getRecordReader
(EmailContentFormat.java:20)
             at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321)
             at org.apache.hadoop.mapred.Child.main(Child.java:155)

I also tried to browse these files through the HDFS web interface,
java.io.EOFException is returned.

Is there any way to recover the files?

Thanks very much,

YY
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message