hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8470) Implementation of 4-layer subclass of NetworkTopology (NetworkTopologyWithNodeGroup)
Date Sun, 17 Jun 2012 21:05:42 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393606#comment-13393606
] 

Tsz Wo (Nicholas), SZE commented on HADOOP-8470:
------------------------------------------------

Thanks for the update.

> isNodeGroupAware() will be used in balancer policy change ...

Let's keep isNodeGroupAware() for balancer.  

> InnerNodeWithNodeGroup(String name, String location) constructor is not used. Yes. we
should remove it as well as the same api in its parent class (InnerNode). What do you think?

It would be great if you could clean up the code.  Let's do it separately.

+1 patch looks good.
                
> Implementation of 4-layer subclass of NetworkTopology (NetworkTopologyWithNodeGroup)
> ------------------------------------------------------------------------------------
>
>                 Key: HADOOP-8470
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8470
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: 1.0.0, 2.0.0-alpha
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: HADOOP-8470-NetworkTopology-new-impl-v2.patch, HADOOP-8470-NetworkTopology-new-impl-v3.patch,
HADOOP-8470-NetworkTopology-new-impl-v4.patch, HADOOP-8470-NetworkTopology-new-impl.patch
>
>
> To support the four-layer hierarchical topology shown in attached figure as a subclass
of NetworkTopology, NetworkTopologyWithNodeGroup was developed along with unit tests. NetworkTopologyWithNodeGroup
overriding the methods add, remove, and pseudoSortByDistance were the most relevant to support
the four-layer topology. The method seudoSortByDistance selects the nodes to use for reading
data and sorts the nodes in sequence of node-local, nodegroup-local, rack- local, rack–off.
Another slightly change to seudoSortByDistance is to support cases of separation data node
and node manager: if the reader cannot be found in NetworkTopology tree (formed by data nodes
only), then it will try to sort according to reader's sibling node in the tree.
> The distance calculation changes the weights from 0 (local), 2 (rack- local), 4 (rack-off)
to: 0 (local), 2 (nodegroup-local), 4 (rack-local), 6 (rack-off).
> The additional node group layer should be specified in the topology script or table mapping,
e.g. input 10.1.1.1, output: /rack1/nodegroup1
> A subclass on InnerNode, InnerNodeWithNodeGroup, was also needed to support NetworkTopologyWithNodeGroup.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message