hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8469) Make NetworkTopology class pluggable
Date Tue, 05 Jun 2012 20:57:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289713#comment-13289713
] 

Tsz Wo (Nicholas), SZE commented on HADOOP-8469:
------------------------------------------------

Hi Junping, thanks a lot for the update.  It is much easier to see what have been changed
in InnerNode.  Some comments:

- Let's rename net.topology.class.name to net.topology.impl (which is consistent to other
keys like fs.file.impl.)

- NET_TOPOLOGY_WITH_NODEGROUP is not used.  What is it for?

- Could you change InnerNode.children to List<Node>?  This is a problem on the existing
code.

- Why doGetRack(loc) is needed instead of letting subclasses overriding getRack(loc) directly?

- Let's rename compareParents(..) to isSameParent(..).  You may want to change the javadoc
to "@return true if their parents are equal, ..."

- Please move isOnSameNodeGroup(..) to right after isNodeGroupAware(..) and change the comment
above isOnSameNodeGroup(..) to javadoc.
                
> Make NetworkTopology class pluggable
> ------------------------------------
>
>                 Key: HADOOP-8469
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8469
>             Project: Hadoop Common
>          Issue Type: Sub-task
>    Affects Versions: 1.0.0, 2.0.0-alpha
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: HADOOP-8469-NetworkTopology-pluggable-v2.patch, HADOOP-8469-NetworkTopology-pluggable-v3.patch,
HADOOP-8469-NetworkTopology-pluggable.patch
>
>
> The class NetworkTopology is where the three-layer hierarchical topology is modeled in
the current code base and is instantiated directly by the DatanodeManager and Balancer.
> To support alternative topologies, changes were make the topology class pluggable, that
is to support using a user specified topology class specified in the Hadoop configuration
file core-defaul.xml. The user specified topology class is instantiated using reflection in
the same manner as other customizable classes in Hadoop. If no use specified topology class
is found, the fallback is to use the NetworkTopology to preserve current behavior. To make
it possible to reuse code in NetworkTopology several minor changes were made to make the class
more extensible. The NetworkTopology class is currently annotated with @InterfaceAudience.LimitedPrivate({"HDFS",
"MapReduce"}) and @InterfaceStability.Unstable.
> The proposed changes in NetworkTopology listed below
> 1. Some fields were changes from private to protected
> 2. Added some protected methods so that sub classes could override behavior
> 3. Added a new method,isNodeGroupAware,to NetworkTopology
> 4. The inner class InnerNode was made a package protected class to it would be easier
to subclass

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message