hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo \(Nicholas\), Sze" <s29752-hadoopu...@yahoo.com>
Subject Re: Zero file size after hsync
Date Fri, 18 Mar 2011 16:55:15 GMT
Hi Viliam,

Which version of Hadoop are you using?

First of all, hsyn is the same as hflush in 0.21 and above.  hflush/hsync won't 
update the file length on the NameNode.  So the answer to your question is yes.  
We have to call DFSDataInputStream.getVisibleLength() to get the visible length 
of the file.

When is the SequenceFile opened?  Before or after hflush/hsync?  Note that only 
new reader can see the new data.  So if the file, normal file or SequenceFile, 
is opened before hflush/hsync, we have to re-open the file in order to see the 
new data.

Anyway, please feel free to file a JIRA if you feel it is a bug or you like to 
have a feature request.

Hope it helps.

From: Viliam Holub <viliam.holub@ucd.ie>
To: hdfs-user@hadoop.apache.org
Sent: Fri, March 18, 2011 9:29:32 AM
Subject: Zero file size after hsync

Hi all,

size of a newly created file is reported to be zero even though I've written
some data and hsync-ed them. Is that correct and expected effect?
hadoop fs -cat will retrieve the data correctly.

As a consequence SequenceFile fails to seek in the file since it tests the
position against file size. And data are there...

View raw message