hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9314) Improve BlockPlacementPolicyDefault's picking of excess replicas
Date Mon, 23 Nov 2015 01:42:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15021350#comment-15021350
] 

Walter Su commented on HDFS-9314:
---------------------------------

Suppose {{first}}, {{second}} are not empty.

Before changes to default policy:
1. choose from {{first}}.

After changes to default policy:
1. choose from {{first}}.
2. If step.1 returns null, fallback to choose from {{second}}.

Suppose first, second are not empty, and suppose in {{first}} there are nodes on same node-group
Before changes to node-group policy:
1. choose from nodes belonging to same node-group from {{first}}.

After changes to node-group policy (What I thought I should be):
1. choose from nodes belonging to same node-group from {{first}}.
2. If step.1 returns null, fallback to all {{first}}.
3. If step.2 returns null, fallback to choose from {{second}}.
(The step.2~3 is the same as default policy)

_Satisfying the placement policy is the prerequisite for the fallback strategy._

What I meant is, we support fallback in default policy, why not node-group policy?
Besides, the 004 patch simply returns {{all}}, there is no preference to {{first}} or {{second}}.
We can't say it's a fallback strategy.

> Improve BlockPlacementPolicyDefault's picking of excess replicas
> ----------------------------------------------------------------
>
>                 Key: HDFS-9314
>                 URL: https://issues.apache.org/jira/browse/HDFS-9314
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Ming Ma
>            Assignee: Xiao Chen
>         Attachments: HDFS-9314.001.patch, HDFS-9314.002.patch, HDFS-9314.003.patch, HDFS-9314.004.patch,
HDFS-9314.005.patch
>
>
> The test case used in HDFS-9313 identified NullPointerException as well as the limitation
of excess replica picking. If the current replicas are on {SSD(rack r1), DISK(rack 2), DISK(rack
3), DISK(rack 3)} and the storage policy changes to HOT_STORAGE_POLICY_ID, BlockPlacementPolicyDefault's
won't be able to delete SSD replica.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message