hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11530) Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault
Date Thu, 30 Mar 2017 18:49:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949590#comment-15949590
] 

Chen Liang commented on HDFS-11530:
-----------------------------------

Thanks [~linyiqun] for looking into it!

I would definitely want to have maybe another JIRA to keep track of the failed tests. We might
also be able to find the other places we need to change for {{DFSNetworkTopology}} when fixing
the tests.

Regarding the other subclass of {{NetworkTopology}}, I think there is only one such class,
that is {{NetworkTopologyWithNodeGroup}}. Currently this class is only used by {{BlockPlacementPolicyWithNodeGroup}}.
Which makes this a rather self-contained change. I think we can just ignore it for now and
revisit this later on if we do want to integrate this performance improvement into {{BlockPlacementPolicyWithNodeGroup}}.

Any other comments [~arpitagarwal]?

> Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault
> ------------------------------------------------------------------------------------
>
>                 Key: HDFS-11530
>                 URL: https://issues.apache.org/jira/browse/HDFS-11530
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Yiqun Lin
>            Assignee: Yiqun Lin
>         Attachments: HDFS-11530.001.patch, HDFS-11530.002.patch
>
>
> The work for {{chooseRandomWithStorageType}} has been merged in HDFS-11482. But this
method is contained in new topology {{DFSNetworkTopology}} which is specified for HDFS. We
should update this and let {{BlockPlacementPolicyDefault}} use the new way since the original
way is inefficient.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message