hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kris Jirapinyo <kjirapi...@biz360.com>
Subject Intra-datanode balancing?
Date Tue, 25 Aug 2009 19:51:27 GMT
Hi all,
    I know this has been filed as a JIRA improvement already
http://issues.apache.org/jira/browse/HDFS-343, but is there any good
workaround at the moment?  What's happening is I have added a few new EBS
volumes to half of the cluster, but Hadoop doesn't want to write to them.
When I try to do cluster rebalancing, since the new disks make the
percentage used lower, it fills up the first two existing local disks, which
is exactly what I don't want to happen.  Currently, I just delete several
subdirs from dfs, since I know that with a replication factor of 3, it'll be
ok, so that fixes the problems in the short term.  But I still cannot get
Hadoop to use those new larger disks efficiently.  Any thoughts?

-- Kris.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message