hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-196) File length not reported correctly after application crash
Date Mon, 21 Jul 2014 19:38:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069112#comment-14069112
] 

Tsz Wo Nicholas Sze commented on HDFS-196:
------------------------------------------

> ... getFileSatus() will return the correct length ...

Oops, it should be "... getFileSatus() will NOT return the correct length ...".

> File length not reported correctly after application crash
> ----------------------------------------------------------
>
>                 Key: HDFS-196
>                 URL: https://issues.apache.org/jira/browse/HDFS-196
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Doug Judd
>
> Our application (Hypertable) creates a transaction log in HDFS.  This log is written
with the following pattern:
> out_stream.write(header, 0, 7);
> out_stream.sync()
> out_stream.write(data, 0, amount);
> out_stream.sync()
> [...]
> However, if the application crashes and then comes back up again, the following statement
> length = mFilesystem.getFileStatus(new Path(fileName)).getLen();
> returns the wrong length.  Apparently this is because this method fetches length information
from the NameNode which is stale.  Ideally, a call to getFileStatus() would return the accurate
file length by fetching the size of the last block from the primary datanode.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message