hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11998) Enable DFSNetworkTopology as default
Date Wed, 21 Jun 2017 00:11:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056705#comment-16056705
] 

Chen Liang commented on HDFS-11998:
-----------------------------------

{{TestAvailableSpaceBlockPlacementPolicy}} fails because {{AvailableSpaceBlockPlacementPolicy}}
only rewrites chooseDataNode(...) for it's optimization purpose. However the caller of this
function in {{BlockPlacementPolicyDefault#chooseRandom()}} has changed to call chooseDataNode(…,
type) on {{DFSNetworkTopology}}, this bypasses the change that AvailableSpaceBlockPlacementPolicy
has made in chooseDataNode(...), thus invalidated its change. Need to also overwrite chooseDataNode(...,
type)  in AvailableSpaceBlockPlacementPolicy. I've made the change locally which seems to
have solved the test failure. Will post next patch after the other test failure gets resolved.

> Enable DFSNetworkTopology as default
> ------------------------------------
>
>                 Key: HDFS-11998
>                 URL: https://issues.apache.org/jira/browse/HDFS-11998
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Chen Liang
>            Assignee: Chen Liang
>         Attachments: HDFS-11998.001.patch
>
>
> HDFS-11530 has made it configurable to use {{DFSNetworkTopology}}, and still uses {{NetworkTopology}}
as default. 
> Given the stress testing in HDFS-11923 which shows the correctness of DFSNetworkTopology,
and the performance testing in HDFS-11535 which shows how DFSNetworkTopology can outperform
NetworkTopology. I think we are at the point where I can and should enable DFSNetworkTopology
as default.
> Any comments/thoughts are more than welcome!



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message