hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Judd (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-5157) File length not reported correctly after application crash
Date Mon, 02 Feb 2009 22:13:59 GMT
File length not reported correctly after application crash
----------------------------------------------------------

                 Key: HADOOP-5157
                 URL: https://issues.apache.org/jira/browse/HADOOP-5157
             Project: Hadoop Core
          Issue Type: Bug
          Components: dfs
    Affects Versions: 0.19.0
            Reporter: Doug Judd
             Fix For: 0.20.0


Our application (Hypertable) creates a transaction log in HDFS.  This log is written with
the following pattern:

out_stream.write(header, 0, 7);
out_stream.sync()
out_stream.write(data, 0, amount);
out_stream.sync()
[...]

However, if the application crashes and then comes back up again, the following statement

length = mFilesystem.getFileStatus(new Path(fileName)).getLen();

returns the wrong length.  Apparently this is because this method fetches length information
from the NameNode which is stale.  Ideally, a call to getFileStatus() would return the accurate
file length by fetching the size of the last block from the primary datanode.



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message