hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8647) Abstract BlockManager's rack policy into BlockPlacementPolicy
Date Thu, 03 Dec 2015 02:07:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037076#comment-15037076
] 

Xiao Chen commented on HDFS-8647:
---------------------------------

Summary from branch-2.7 backport to branch-2.6 below:
- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
(HDFS-7411 which is in 2.7 but not 2.6. Resolved by copying over 2.7 but removing changes
related to HDFS-7411. Also multiple branch2.6 fixes: 1. Changed a couple of initializing statements
to fix the error 'diamond operator is not supported in -source 1.6'. 2. Updated log statement
in BlockManager.java#2928 to not use placeholder syntax.
- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
(import conflicts.)
- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
(import conflicts. Also fixed diamond operators  to compile.)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
(The test {{TestBalancer#testBalancerWithPinnedBlocks}} from HDFS-6133 in 2.7 but not in 2.6.
Resolved by not bringing in.)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
1 line conflict in HDFS-9083’s new test
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
(import conflicts. Also fixed diamond operators  to compile.)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
(import conflicts.)

> Abstract BlockManager's rack policy into BlockPlacementPolicy
> -------------------------------------------------------------
>
>                 Key: HDFS-8647
>                 URL: https://issues.apache.org/jira/browse/HDFS-8647
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ming Ma
>            Assignee: Brahma Reddy Battula
>             Fix For: 2.8.0
>
>         Attachments: HDFS-8647-001.patch, HDFS-8647-002.patch, HDFS-8647-003.patch, HDFS-8647-004.patch,
HDFS-8647-004.patch, HDFS-8647-005.patch, HDFS-8647-006.patch, HDFS-8647-007.patch, HDFS-8647-008.patch,
HDFS-8647-009.patch, HDFS-8647-branch26.patch, HDFS-8647-branch27.patch
>
>
> Sometimes we want to have namenode use alternative block placement policy such as upgrade
domains in HDFS-7541.
> BlockManager has built-in assumption about rack policy in functions such as useDelHint,
blockHasEnoughRacks. That means when we have new block placement policy, we need to modify
BlockManager to account for the new policy. Ideally BlockManager should ask BlockPlacementPolicy
object instead. That will allow us to provide new BlockPlacementPolicy without changing BlockManager.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message