hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Intra-datanode balancing?
Date Tue, 25 Aug 2009 20:16:23 GMT
Change the ordering of the volumes in the ocnfig files.

On Tue, Aug 25, 2009 at 12:51 PM, Kris Jirapinyo <kjirapinyo@biz360.com>wrote:

> Hi all,
>    I know this has been filed as a JIRA improvement already
> http://issues.apache.org/jira/browse/HDFS-343, but is there any good
> workaround at the moment?  What's happening is I have added a few new EBS
> volumes to half of the cluster, but Hadoop doesn't want to write to them.
> When I try to do cluster rebalancing, since the new disks make the
> percentage used lower, it fills up the first two existing local disks,
> which
> is exactly what I don't want to happen.  Currently, I just delete several
> subdirs from dfs, since I know that with a replication factor of 3, it'll
> be
> ok, so that fixes the problems in the short term.  But I still cannot get
> Hadoop to use those new larger disks efficiently.  Any thoughts?
> -- Kris.

Ted Dunning, CTO

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message