hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8820) Backport HADOOP-8469 and HADOOP-8470: add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup)
Date Tue, 13 Nov 2012 09:17:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496055#comment-13496055
] 

Jing Zhao commented on HADOOP-8820:
-----------------------------------

bq. The 1st is not a issue as children is already checked before.
I'm not sure if I understand the code correctly here. So the isRack() function first checks
the children of the current node ("if (getChildren().isEmpty())"), but does not check the
children of the first child of the current node. Thus is it possible that "((InnerNode) firstChild).children"
is empty (in which case the following get(0) may cause an IndexOutofBoundsException)? Or somewhere
outside the function it has been checked?

By the way, in the latest patch in HADOOP-8817 (HADOOP-8817-v3.patch), several method definitions
seem to be repeated between BlockPlacementPolicy.java and BlockPlacementPolicyDefault.java
(e.g., adjustSetsWithChosenReplica(), splitNodesWithLocalityGroup(), getLocalityGroupForSplit()
and getRack()). So do we need to recheck the patch to verify?
                
> Backport HADOOP-8469 and HADOOP-8470: add "NodeGroup" layer in new NetworkTopology (also
known as NetworkTopologyWithNodeGroup)
> -------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-8820
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8820
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: net
>    Affects Versions: 1.0.0
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: HADOOP-8820.b1.002.patch, HADOOP-8820.b1.003.patch, HADOOP-8820.patch
>
>
> This patch backport HADOOP-8469 and HADOOP-8470 to branch-1 and includes:
> 1. Make NetworkTopology class pluggable for extension.
> 2. Implement a 4-layer NetworkTopology class (named as NetworkTopologyWithNodeGroup)
to use in virtualized environment (or other situation with additional layer between host and
rack).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message