hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Collins <...@cloudera.com>
Subject Re: Adding hard-disks to an existing HDFS cluster
Date Sun, 28 Feb 2010 22:14:37 GMT
Hey Oded,

You don't need to format to to add disks to DNs. Just format them and
add the directories to dfs.data.dir in the config file, and restart
the DN. The data in an individual DN won't be automatically balanced
across disks when you restart. Rebalancing is not necessary as the DN
will round robin blocks over all disks and stop writing to a disk when
it fills. If you want the disks to be balanced you can do that
manually by copying the block files from the existing data directories
to the new ones--HDFS just checks for the blocks at startup, it
doesn't keep track of which directory they were stored in.

Thanks,
Eli

On Sun, Feb 28, 2010 at 1:45 PM, Oded Rosen <oded@legolas-media.com> wrote:
> We have an existing HDFS cluster with several datanodes, and we want to add
> each of the datanodes another hard-disk, as an addition to the existing
> ones.
> Is there a way of doing this without formatting the cluster? Our aim is to
> save all the data where it is, add and configure the new disks, perform a
> balance - with no format whatsoever.
> Is it possible? if so, how?
>
> Any kind of help will be welcomed.
> Thanks,
>
> --
> Oded
>

Mime
View raw message