hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-196) File length not reported correctly after application crash
Date Mon, 21 Jul 2014 19:30:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069093#comment-14069093

Allen Wittenauer commented on HDFS-196:


I'm tempted to close this as stale, but it would be good for someone more familiar with the
issue to do that.

> File length not reported correctly after application crash
> ----------------------------------------------------------
>                 Key: HDFS-196
>                 URL: https://issues.apache.org/jira/browse/HDFS-196
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Doug Judd
> Our application (Hypertable) creates a transaction log in HDFS.  This log is written
with the following pattern:
> out_stream.write(header, 0, 7);
> out_stream.sync()
> out_stream.write(data, 0, amount);
> out_stream.sync()
> [...]
> However, if the application crashes and then comes back up again, the following statement
> length = mFilesystem.getFileStatus(new Path(fileName)).getLen();
> returns the wrong length.  Apparently this is because this method fetches length information
from the NameNode which is stale.  Ideally, a call to getFileStatus() would return the accurate
file length by fetching the size of the last block from the primary datanode.

This message was sent by Atlassian JIRA

View raw message