hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Beaudreault <bbeaudrea...@hubspot.com>
Subject Re: Misconfiguration of hdfs-site.xml
Date Tue, 18 Dec 2012 20:36:25 GMT
Updating the config and restarting will result in data loss.  You're going
to want to migrate the data from your tmp directory to the appropriate

The NameNode will use the dfs.name.dir property, and DataNodes will use the
dfs.data.dir.  Assuming you aren't running a DN and NN on the same server
(don't do that), then they shouldn't collide/overlap.  Even still its
probably best to use different directories.  For instance mine is like this:

dfs.name.dir = /mnt/hdfs/name,/mnt2/hdfs/name
= /mnt/hdfs/data,/mnt2/hdfs/data,/mnt3/hdfs/data,/mnt4/hdfs/data

Hope this helps.

On Tue, Dec 18, 2012 at 3:28 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> Hi,
> For months now I'm using my hadoop cluster with absolutly nothing
> related to the drive directory on my hdfs-site.xml file.
> It seems that it's using the hadoop.tmp.dir directory to store data.
> My hadoop.tmp.dir is pointing to
> /home/hadoop/haddop_drive/${user.name} and on my
> /home/hadoop/haddop_drive directory I can see hadoop and hbase.
> Now, I want to configure that properly without loosing the data I have.
> I'm using 1.0.3.
> Based on the documentation
> http://hadoop.apache.org/docs/r1.0.3/cluster_setup.html#Site+Configuration
> it seems I need to setup dfs.data.dir to point to
> /home/hadoop/haddop_drive. If I add this entry and restart my
> datanode, will it "simply" continue to point to the same place and
> find the data he needs?
> Also, what about dfs.name.dir ? I guess I can simply point it to the
> same place, right?
> On  /home/hadoop/haddop_drive/hadoop/dfs I have data and name
> directories. So I'm not sure if I should point the properties above to
>  /home/hadoop/haddop_drive or to  /home/hadoop/haddop_drive/dfs.
> Thanks,
> JM

View raw message