Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 30 Mar 2017 18:49:41 +0000 (UTC)
From: "Chen Liang (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13055893.1489486130000.168054.1490899781829@Atlassian.JIRA>
In-Reply-To: <JIRA.13055893.1489486130000@Atlassian.JIRA>
References: <JIRA.13055893.1489486130000@Atlassian.JIRA> <JIRA.13055893.1489486130156@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-11530) Use HDFS specific network topology
 to choose datanode in BlockPlacementPolicyDefault
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 30 Mar 2017 18:49:51 -0000


    [ https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949590#comment-15949590 ] 

Chen Liang commented on HDFS-11530:
-----------------------------------

Thanks [~linyiqun] for looking into it!

I would definitely want to have maybe another JIRA to keep track of the failed tests. We might also be able to find the other places we need to change for {{DFSNetworkTopology}} when fixing the tests.

Regarding the other subclass of {{NetworkTopology}}, I think there is only one such class, that is {{NetworkTopologyWithNodeGroup}}. Currently this class is only used by {{BlockPlacementPolicyWithNodeGroup}}. Which makes this a rather self-contained change. I think we can just ignore it for now and revisit this later on if we do want to integrate this performance improvement into {{BlockPlacementPolicyWithNodeGroup}}.

Any other comments [~arpitagarwal]?

> Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault
> ------------------------------------------------------------------------------------
>
>                 Key: HDFS-11530
>                 URL: https://issues.apache.org/jira/browse/HDFS-11530
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Yiqun Lin
>            Assignee: Yiqun Lin
>         Attachments: HDFS-11530.001.patch, HDFS-11530.002.patch
>
>
> The work for {{chooseRandomWithStorageType}} has been merged in HDFS-11482. But this method is contained in new topology {{DFSNetworkTopology}} which is specified for HDFS. We should update this and let {{BlockPlacementPolicyDefault}} use the new way since the original way is inefficient.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org