hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2021) HDFS Junit test TestWriteRead failed with inconsistent visible length of a file
Date Wed, 01 Jun 2011 17:17:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042298#comment-13042298

Tsz Wo (Nicholas), SZE commented on HDFS-2021:

Daryn, I agree that we need {{replyAck.isSuccess()}}.

That said, I'm a bit confused about why a datanode updates its bytesAcked iff all downstreams
are successful. ...  bytesAcked is intended to track exactly how many bytes were written throughout
the entire pipeline ...

You are totally correct that it is the intention; see Section 3.3 in the [Append Design Doc|https://issues.apache.org/jira/secure/attachment/12445209/appendDesign3.pdf]
in HDFS-265.

> HDFS Junit test TestWriteRead failed with inconsistent visible length of a file 
> --------------------------------------------------------------------------------
>                 Key: HDFS-2021
>                 URL: https://issues.apache.org/jira/browse/HDFS-2021
>             Project: Hadoop HDFS
>          Issue Type: Bug
>         Environment: Linux RHEL5
>            Reporter: CW Chung
>            Assignee: John George
>            Priority: Minor
>         Attachments: HDFS-2021.patch
> The junit test failed when iterates a number of times with larger chunk size on Linux.
Once a while, the visible number of bytes seen by a reader is slightly less than what was
supposed to be. 
> When run with the following parameter, it failed more often on Linux ( as reported by
John George) than my Mac:
>   private static final int WR_NTIMES = 300;
>   private static final int WR_CHUNK_SIZE = 10000;
> Adding more debugging output to the source, this is a sample of the output:
> Caused by: java.io.IOException: readData mismatch in byte read: expected=2770000 ; got
>         at org.apache.hadoop.hdfs.TestWriteRead.readData(TestWriteRead.java:141)

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message