hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8204) Mover/Balancer should not schedule two replicas to the same DN
Date Sat, 25 Apr 2015 06:03:39 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Walter Su updated HDFS-8204:
----------------------------
    Description: 
Balancer moves blocks between Datanode(Ver. <2.6 ).
Balancer moves blocks between StorageGroups ( introduced by HDFS-6584) , in the new version(Ver.
>=2.6) .
function
{code}
class DBlock extends Locations<StorageGroup>
DBlock.isLocatedOn(StorageGroup loc)
{code}
is flawed, may causes 2 replicas ends in same node after running balance.

For example:
We have 2 nodes. Each node has two storages.
We have (DN0, SSD), (DN0, DISK), (DN1, SSD), (DN1, DISK).
We have a block with ONE_SSD storage policy.
The block has 2 replicas. They are in (DN0,SSD) and (DN1,DISK).
Replica in (DN0,SSD) should not be moved to (DN1,SSD) after running Balancer.
Otherwise DN1 has 2 replicas.

  was:
Balancer moves blocks between Datanode(Ver. <2.6 ).
Balancer moves blocks between StorageGroups ( introduced by HDFS-6584) , in the new version(Ver.
>=2.6) .
function
{code}
class DBlock extends Locations<StorageGroup>
DBlock.isLocatedOn(StorageGroup loc)
{code}
is flawed, may causes 2 replicas ends in same node after running balance.


> Mover/Balancer should not schedule two replicas to the same DN
> --------------------------------------------------------------
>
>                 Key: HDFS-8204
>                 URL: https://issues.apache.org/jira/browse/HDFS-8204
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: balancer & mover
>            Reporter: Walter Su
>            Assignee: Walter Su
>         Attachments: HDFS-8204.001.patch, HDFS-8204.002.patch
>
>
> Balancer moves blocks between Datanode(Ver. <2.6 ).
> Balancer moves blocks between StorageGroups ( introduced by HDFS-6584) , in the new version(Ver.
>=2.6) .
> function
> {code}
> class DBlock extends Locations<StorageGroup>
> DBlock.isLocatedOn(StorageGroup loc)
> {code}
> is flawed, may causes 2 replicas ends in same node after running balance.
> For example:
> We have 2 nodes. Each node has two storages.
> We have (DN0, SSD), (DN0, DISK), (DN1, SSD), (DN1, DISK).
> We have a block with ONE_SSD storage policy.
> The block has 2 replicas. They are in (DN0,SSD) and (DN1,DISK).
> Replica in (DN0,SSD) should not be moved to (DN1,SSD) after running Balancer.
> Otherwise DN1 has 2 replicas.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message