hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8789) Block Placement policy migrator
Date Fri, 04 Mar 2016 22:23:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15180649#comment-15180649
] 

Ming Ma commented on HDFS-8789:
-------------------------------

Maybe we can use migrator as the first scenario for doing block movement scheduling inside
namenode? Changing balancer from client-side to server-side requires more work to make the
command line backward compatible given it is used by admins and automation. Migrator is a
brand new tool and we can use it as the starting point for the proper design to accommodate
migrator, balancer and other scenarios done inside namenode.

Other useful statistics provided by the migrator tool such as block size distribution, block
rack diversity distribution and block replication distribution is somewhat independent of
migrator. Maybe we can have another tool to generate those stats, or maybe fsck.

> Block Placement policy migrator
> -------------------------------
>
>                 Key: HDFS-8789
>                 URL: https://issues.apache.org/jira/browse/HDFS-8789
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Chris Trezzo
>            Assignee: Chris Trezzo
>         Attachments: HDFS-8789-trunk-STRAWMAN-v1.patch
>
>
> As we start to add new block placement policies to HDFS, it will be necessary to have
a robust tool that can migrate HDFS blocks between placement policies. This jira is for the
design and implementation of that tool.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message