Mailing-List: contact hdfs-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of dariusz.czerski@gmail.com
 designates 209.85.215.174 as permitted sender)
MIME-Version: 1.0
Sender: dariusz.czerski@gmail.com
Date: Wed, 10 Aug 2011 17:01:07 +0200
Message-ID: 
 <CANCoYZv0a0yK2EHSF7NU3QwnyvFyhLMC+KQa6T=gWq8FP+4BUA@mail.gmail.com>
Subject: Cannot read from file after server crash
From: Dariusz Czerski <dariusz.czerski@ipipan.waw.pl>
To: hdfs-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hi,

I have a hadoop cluster of 5 servers, version 0.21.0.
On one of the servers i have a program which is writing to some file
in HDFS, calling hflush and hsync after every write operation.
When this server crashed (for example by power off) killing both
DataNode and my little program,
the data written to the file cannot be read for some time by another client.
Duplication factor is set to 3.
Situation changes after some time (several minutes), as i susspect
NameNode needs time to proceed file recovering.
When i try to append to this broken file, the
AlreadyBeingCreatedException or RecoveryInProgressException exception
occurring, but at the same time reading from the file gets no data
(without any exeption).

Is this situation correct?
Is there any way to check that this file is in the recovery state? If
yes, program can wait some time for recovery finish.

Thanks
Darek C