hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10652) Add a unit test for HDFS-4660
Date Wed, 20 Jul 2016 23:20:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386792#comment-15386792
] 

Yongjun Zhang commented on HDFS-10652:
--------------------------------------

Thanks a lot for the updated rev [~vinayrpet]!

I'm uploading an updated rev (003) for ease for commenting:

1, In the patch, we inject an error when detecting the condition (ackSize % 512 > 0 &&
ackSize < diskSize) at the second DN, the third DN is thrown away and we add a new DN.
In pipeline recovery, when we do block transfer to the new DN, the copy source may be the
first or the second DN. It's not deterministic in this test. I did quite a few rounds of test,
and saw that using either one as a source seems fine, it demonstrates the same ackSize and
diskSize satisfying (ackSize % 512 > 0 && ackSize < diskSize), so this is good.
However, I'd like to point this out, since in the real case, the copy source is the DN that
satisfies the above mentioned condition.

2. I replaced the hardcoded constants with constants, as 
{code}
    final int CHUNK_SIZE = 512;
    final int ONE_WRITE_SIZE = 5000;
    final int TOTAL_SIZE = 2 * 1024 * 1024;
    final int ERROR_INJECTION_LOC = TOTAL_SIZE / 2;
{code}
I thought the {{TOTAL_SIZE}} doesn't have to be 1M, and  {{ERROR_INJECTION_LOC}} doesn't have
to be half the total size, so I did the following change
{code}
    final int CHUNK_SIZE = 512;
    final int ONE_WRITE_SIZE = 5000;
    final int TOTAL_SIZE = 1024 * 1024;
    final int ERROR_INJECTION_LOC = 512;
{code}
and that does work too.

Would you please take a look? 

Thanks.




> Add a unit test for HDFS-4660
> -----------------------------
>
>                 Key: HDFS-10652
>                 URL: https://issues.apache.org/jira/browse/HDFS-10652
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode, hdfs
>            Reporter: Yongjun Zhang
>            Assignee: Vinayakumar B
>         Attachments: HDFS-10652-002.patch, HDFS-10652.001.patch, HDFS-10652.003.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message