hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "FAQ" by KonstantinShvachko
Date Thu, 16 Nov 2006 22:56:31 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by KonstantinShvachko:
http://wiki.apache.org/lucene-hadoop/FAQ

------------------------------------------------------------------------------
  
  If you find something that you wish were done better, and know how to fix it, read HowToContribute,
and contribute a patch.
  
+ == 5. If I add new data-nodes to the cluster will HDFS move the blocks to the newly added
nodes in order to balance disk space utilization between the nodes? ==
+ 
+ No, HDFS will not move blocks to new nodes automatically. However, newly created files will
likely have their blocks placed on the new nodes.
+ 
+ There are several ways to rebalance the cluster manually.
+  1. Select a subset of files that take up a good percentage of your disk space; copy them
to new locations in HDFS; remove the old copies of the files; rename the new copies to their
original names.
+  2. A simpler way, with no interruption of service, is to turn up the replication of files,
wait for transfers to stabilize, and then turn the replication back down.
+  3. Yet another way to re-balance blocks is to turn off the data-node, which is full, wait
until its blocks are replicated, and then bring it back again. The over-replicated blocks
will be randomly removed from different nodes, so you really get them rebalanced not just
removed from the current node.
+ 

Mime
View raw message