hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: How to redistribute files on HDFS after adding new machines to cluster?
Date Sat, 08 Aug 2009 05:42:06 GMT
I think that I remember that you essentially doubled your storage before
starting balancing.

This means that about 1 TB will need to be copied.  By default the balancer
only moves 1MB/s (per node, I believe).  This means that it will take a LONG
time to balance your cluster.  You can increase this speed limit, but there
isn't usually much need to do so.  Running the balancer while using your
cluster is generally not a big deal since the balancer consumes so little

On Fri, Aug 7, 2009 at 10:10 PM, prashant ullegaddi <
prashullegaddi@gmail.com> wrote:

> *   Capacity : 3.93 TB DFS Remaining : 2.11 TB DFS Used : 1.31 TB DFS
> Used%:33.44 % Live
> Nodes <http://megh01:50070/dfshealth.jsp#LiveNodes> : 10 Dead
> Nodes<http://megh01:50070/dfshealth.jsp#DeadNodes>
> : 0
> If I interrupt it now, what will happen? I've to run a job now. I think
> balancing and running a job
> may not happen together as one will slow down the other.

Ted Dunning, CTO

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message