hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Chansler (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-3211) Utility to enforce current replication policy on already exitsting blocks
Date Tue, 08 Apr 2008 19:05:24 GMT
Utility to enforce current replication policy on already exitsting blocks
-------------------------------------------------------------------------

                 Key: HADOOP-3211
                 URL: https://issues.apache.org/jira/browse/HADOOP-3211
             Project: Hadoop Core
          Issue Type: New Feature
            Reporter: Robert Chansler


If the replica placement policy changes, or if the cluster topology changes, existing block
replicas will not be deliberately moved to implement the new policy or to reflect the new
topology. The re-replicator and the re-balancer always implement the current policy, but a
healthy block may never attract the attention of either of those facilities. 

In practice, all replicas of a block have been found within a single rack (as allowed by the
previous _random rack_ placement policy. The {{fsck}} facility reports such examples.

The re-placer utility could follow the model of the re-balancer. Or even be part of the re-balancer.

An alternative implementation would scan the {{fsck}} report for curious blocks, and rename
a replica block file on the data node, so as to hide it. The re-replicator would be expected
to create a new replica properly placed. (This is a fix up that any administrator could implement.)
(1761414)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message