hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6261) Document for enabling node group layer in HDFS
Date Tue, 02 Jun 2015 12:25:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569020#comment-14569020
] 

Junping Du commented on HDFS-6261:
----------------------------------

Thanks [~decster] for updating the patch and [~aw] for review comments!
The latest patch get much closer, some additional comments:
{code}
+<property>
+  <name>net.topology.nodegroup.aware</name>
+  <value>true</value>
+</property>
{code}
This property is only used in branch-1 (for identify jobtracker's awareness of nodegroup),
but not used in trunk and branch-2. Let's remove it.

{code}
+  <name>dfs.block.replicator.classname</name>
+  <value>
+    org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicyWithNodeGroup
+  </value>
{code}
The package name for BlockPlacementPolicyWithNodeGroup is already be refactor to "org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyWithNodeGroup".
Let's do the update.

{code}
+    The remaining replicas are placed randomly across other nodes.
{code}
Better to mention that this random won't conflict policy no. 1: no two replicas will be placed
among nodes within the same nodegroup.

Other looks fine to me.

> Document for enabling node group layer in HDFS
> ----------------------------------------------
>
>                 Key: HDFS-6261
>                 URL: https://issues.apache.org/jira/browse/HDFS-6261
>             Project: Hadoop HDFS
>          Issue Type: Task
>          Components: documentation
>            Reporter: Wenwu Peng
>            Assignee: Binglin Chang
>              Labels: documentation
>         Attachments: 2-layer-topology.png, 3-layer-topology.png, 3layer-topology.png,
4layer-topology.png, HDFS-6261.004.patch, HDFS-6261.005.patch, HDFS-6261.006.patch, HDFS-6261.007.patch,
HDFS-6261.008.patch, HDFS-6261.009.patch, HDFS-6261.v1.patch, HDFS-6261.v1.patch, HDFS-6261.v2.patch,
HDFS-6261.v3.patch
>
>
> Most of patches from Umbrella JIRA HADOOP-8468  have committed, However there is no site
to introduce NodeGroup-aware(HADOOP Virtualization Extensisons) and how to do configuration.
so we need to doc it.
> 1.  Doc NodeGroup-aware relate in http://hadoop.apache.org/docs/current 
> 2.  Doc NodeGroup-aware properties in core-default.xml.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message