hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom White <...@cloudera.com>
Subject Re: Rebalancing Hadoop Cluster running 15.3
Date Thu, 25 Jun 2009 09:46:57 GMT
Hi Usman,

Before the rebalancer was introduced one trick people used was to
increase the replication on all the files in the system, wait for
re-replication to complete, then decrease the replication to the
original level. You can do this using hadoop fs -setrep.

Cheers,
Tom

On Thu, Jun 25, 2009 at 10:33 AM, Usman Waheed<usmanw@opera.com> wrote:
> Hi,
>
> One of our test clusters is running HADOOP 15.3 with replication level set
> to 2. The datanodes are not balanced at all.
>
> Datanode_1: 52%
> Datanode_2: 82%
> Datanode_3: 30%
>
> 15.3 does not have the rebalancer capability, we are planning to upgrade but
> not for now.
>
> If i take out Datanode_1 from the cluster (decommission for sometime) will
> HADOOP balance so that Datanode_2 and Datanode_3 will even out to 56%?
> Then i can re-introduce Datanode_1 back into the cluster.
>
> Comments/Suggestions please?
>
> Thanks,
> Usman
>

Mime
View raw message