hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11659) TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery.
Date Thu, 04 May 2017 04:31:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996163#comment-15996163

Yongjun Zhang commented on HDFS-11659:

Hi [~eddyxu],

Thanks for working on this issue. 

I did a quick review, one thought, instead of adding a DN later, can we start a cluster of
4 DNs (which is more natural to me) instead for this test? Or maybe I missed something.


> TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available
for pipeline recovery.
> ----------------------------------------------------------------------------------------------------------------
>                 Key: HDFS-11659
>                 URL: https://issues.apache.org/jira/browse/HDFS-11659
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>    Affects Versions: 2.7.3, 3.0.0-alpha2
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>         Attachments: HDFS-11659.000.patch
> The test fails after the following error messages:
> {code}
> java.io.IOException: Failed to replace a bad datanode on the existing pipeline due to
no more good datanodes being available to try. (Nodes: current=[DatanodeInfoWithStorage[,DS-b4ec61fc-657c-4e2a-9dc3-8d93b7769a2b,DISK],
DatanodeInfoWithStorage[,DS-18bca8d7-048d-4d7f-9594-d2df16096a3d,DISK]], original=[DatanodeInfoWithStorage[,DS-b4ec61fc-657c-4e2a-9dc3-8d93b7769a2b,DISK],
DatanodeInfoWithStorage[,DS-18bca8d7-048d-4d7f-9594-d2df16096a3d,DISK]]). The
current failed datanode replacement policy is DEFAULT, and a client may configure this via
'dfs.client.block.write.replace-datanode-on-failure.policy' in its configuration.
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:1280)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:1354)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1512)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:1236)
> 	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:721)
> {code}
> In such case, the DataNode that has removed can not be used in the pipeline recovery.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message