hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "J.Andreina (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
Date Wed, 16 Sep 2015 09:27:45 GMT

     [ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

J.Andreina updated HDFS-9044:
-----------------------------
    Description: 
Passing Favored nodes intention is to place replica among the favored node

Current behavior in Node group is 
	If favored node is not available it goes to one among favored nodegroup. 
{noformat}
Say for example:
	1)I need 3 replicas and passed 5 favored nodes.
	2)Out of 5 favored nodes 3 favored nodes are not good.
	3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node returned , 3 will
be random node from 3 bad FavoredNode's nodegroup. 
	4)Then there is a probability that all my 3 replicas are placed on Random node from FavoredNodes's
nodegroup , instead of giving priority to 2 favored nodes returned as target.
{noformat}

*Instead of returning 5 targets on 3rd step above , we can return 2 good favored nodes as
target*
*And remaining 1 needed replica can be chosen from Random node of bad FavoredNodes's nodegroup.*
This will make sure that the FavoredNodes are given priority.

  was:
Current behavior:

While choosing targets for replicas if favored nodes are not available , then it goes for
choosing the random node from the favored node group instead of falling back to regular BlockPlacementPolicyWithNodeGroup


> Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
> --------------------------------------------------------------------------------------
>
>                 Key: HDFS-9044
>                 URL: https://issues.apache.org/jira/browse/HDFS-9044
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: J.Andreina
>            Assignee: J.Andreina
>
> Passing Favored nodes intention is to place replica among the favored node
> Current behavior in Node group is 
> 	If favored node is not available it goes to one among favored nodegroup. 
> {noformat}
> Say for example:
> 	1)I need 3 replicas and passed 5 favored nodes.
> 	2)Out of 5 favored nodes 3 favored nodes are not good.
> 	3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node returned , 3
will be random node from 3 bad FavoredNode's nodegroup. 
> 	4)Then there is a probability that all my 3 replicas are placed on Random node from
FavoredNodes's nodegroup , instead of giving priority to 2 favored nodes returned as target.
> {noformat}
> *Instead of returning 5 targets on 3rd step above , we can return 2 good favored nodes
as target*
> *And remaining 1 needed replica can be chosen from Random node of bad FavoredNodes's
nodegroup.*
> This will make sure that the FavoredNodes are given priority.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message