hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic
Date Thu, 14 May 2015 07:01:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14543305#comment-14543305
] 

Walter Su commented on HDFS-7621:
---------------------------------

>...moveBlockAcrossStorage requires Mover send correct numOfBytes, otherwise DN reject
the command.
Problem solved.
Mover parses numBytes of internal block from ECSchema from HdfsFileStatus.
Balancer gets numBytes of internal block from NN which parsed using BlockInfoStriped.

> Erasure Coding: update the Balancer/Mover data migration logic
> --------------------------------------------------------------
>
>                 Key: HDFS-7621
>                 URL: https://issues.apache.org/jira/browse/HDFS-7621
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Jing Zhao
>            Assignee: Walter Su
>              Labels: HDFS-7285
>         Attachments: HDFS-7621.001.patch, HDFS-7621.002.patch, HDFS-7621.003.patch, HDFS-7621.004.patch
>
>
> Currently the Balancer/Mover only considers the distribution of replicas of the same
block during data migration: the migration cannot decrease the number of racks. With EC the
Balancer and Mover should also take into account the distribution of blocks belonging to the
same block group.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message