hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6801) Archival Storage: Add a new data migration tool
Date Mon, 18 Aug 2014 10:38:18 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14100523#comment-14100523
] 

Vinayakumar B commented on HDFS-6801:
-------------------------------------

Patch looks great. 

Small nits here.
 1. Mover.java, {{iteration}} not used.
{code}private ExitStatus run(int iteration) {{code}

2. class Javadoc similar to Balancer may be required explaining how Mover works.

3. And as Jing said in earlier comment, processing entire namespace would take more time in
case of big clusters. 
Can we have optional path arguments to process only those paths for the Mover..? By default
entire namespace is fine.

> Archival Storage: Add a new data migration tool 
> ------------------------------------------------
>
>                 Key: HDFS-6801
>                 URL: https://issues.apache.org/jira/browse/HDFS-6801
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer, namenode
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>         Attachments: h6801_20140813.patch, h6801_20140814.patch, h6801_20140814b.patch,
h6801_20140817.patch
>
>
> The tool is similar to Balancer.  It periodic scans the blocks in HDFS and uses path
and/or other meta data (e.g. mtime) to determine if a block should be cooled down (i.e. hot
=> warm, or warm => cold) or warmed up (i.e. cold => warm, or warm => hot).  In
contrast to Balancer, the migration tool always move replicas to a different storage type.
 Similar to Balancer, the replicas are moved in a way that the number of racks the block does
not decrease.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message