hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8647) Abstract BlockManager's rack policy into BlockPlacementPolicy
Date Tue, 01 Dec 2015 23:58:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034910#comment-15034910
] 

Xiao Chen commented on HDFS-8647:
---------------------------------

Attached patch for branch 2.6. This is cherry-picked from the 2.7 patch and modified upon.
Conflicts:
- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
(import conflicts)
- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
(import conflicts)
- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
(import conflicts)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
(The test is only in branch2.7 by HDFS-6133, so not changed here)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
(1 line conflict in the branch-only test brought in by HDFS-9083)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
(import conflicts)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
(import conflicts)

Both the 2.6 patch and 2.7 patch are compiled locally, and passed the changed unit tests.
Please let me know what further testing/verification is needed for them to qualify. Thanks!

> Abstract BlockManager's rack policy into BlockPlacementPolicy
> -------------------------------------------------------------
>
>                 Key: HDFS-8647
>                 URL: https://issues.apache.org/jira/browse/HDFS-8647
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ming Ma
>            Assignee: Brahma Reddy Battula
>             Fix For: 2.8.0
>
>         Attachments: HDFS-8647-001.patch, HDFS-8647-002.patch, HDFS-8647-003.patch, HDFS-8647-004.patch,
HDFS-8647-004.patch, HDFS-8647-005.patch, HDFS-8647-006.patch, HDFS-8647-007.patch, HDFS-8647-008.patch,
HDFS-8647-009.patch, HDFS-8647-branch27.patch
>
>
> Sometimes we want to have namenode use alternative block placement policy such as upgrade
domains in HDFS-7541.
> BlockManager has built-in assumption about rack policy in functions such as useDelHint,
blockHasEnoughRacks. That means when we have new block placement policy, we need to modify
BlockManager to account for the new policy. Ideally BlockManager should ask BlockPlacementPolicy
object instead. That will allow us to provide new BlockPlacementPolicy without changing BlockManager.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message