hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dariusz Czerski <dariusz.czer...@ipipan.waw.pl>
Subject Cannot read from file after server crash
Date Wed, 10 Aug 2011 15:01:07 GMT

I have a hadoop cluster of 5 servers, version 0.21.0.
On one of the servers i have a program which is writing to some file
in HDFS, calling hflush and hsync after every write operation.
When this server crashed (for example by power off) killing both
DataNode and my little program,
the data written to the file cannot be read for some time by another client.
Duplication factor is set to 3.
Situation changes after some time (several minutes), as i susspect
NameNode needs time to proceed file recovering.
When i try to append to this broken file, the
AlreadyBeingCreatedException or RecoveryInProgressException exception
occurring, but at the same time reading from the file gets no data
(without any exeption).

Is this situation correct?
Is there any way to check that this file is in the recovery state? If
yes, program can wait some time for recovery finish.

Darek C

View raw message