hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: Misconfiguration of hdfs-site.xml
Date Tue, 18 Dec 2012 20:53:04 GMT
> Updating the config and restarting will result in data loss.

Ok. If that's the case, then I can still re-configure the servers one
by one manually and restart them. And moving to the next server only
when the replication is completed.

> dfs.data.dir presently is /home/hadoop/haddop_drive/${user.name}/dfs/data
> dfs.name.dir presently is /home/hadoop/haddop_drive/${user.name}/dfs/name

Ok. That sound like what I was expecting. I don't really want to move
the data/name directories. My goal is just to fix the lack of

I will try to update with the properties above on one of the servers
and restart it. And if it's working fine, I will apply that on all the



2012/12/18, Harsh J <harsh@cloudera.com>:
> Given your previous config is
> "/home/hadoop/haddop_drive/${user.name}", if the defaults are to go
> by, then:
> dfs.data.dir presently is /home/hadoop/haddop_drive/${user.name}/dfs/data
> dfs.name.dir presently is /home/hadoop/haddop_drive/${user.name}/dfs/name
> If you are specifying explicit paths, it will need to be the full one
> (preferably pre-evaluated for ${user.name}) as above, exactly.
> If you wish to move the location somewhere else, you will need to mv
> the {data,name} directories elsewhere and re-point down to that path
> component again.
> On Wed, Dec 19, 2012 at 1:58 AM, Jean-Marc Spaggiari
> <jean-marc@spaggiari.org> wrote:
>> Hi,
>> For months now I'm using my hadoop cluster with absolutly nothing
>> related to the drive directory on my hdfs-site.xml file.
>> It seems that it's using the hadoop.tmp.dir directory to store data.
>> My hadoop.tmp.dir is pointing to
>> /home/hadoop/haddop_drive/${user.name} and on my
>> /home/hadoop/haddop_drive directory I can see hadoop and hbase.
>> Now, I want to configure that properly without loosing the data I have.
>> I'm using 1.0.3.
>> Based on the documentation
>> http://hadoop.apache.org/docs/r1.0.3/cluster_setup.html#Site+Configuration
>> it seems I need to setup dfs.data.dir to point to
>> /home/hadoop/haddop_drive. If I add this entry and restart my
>> datanode, will it "simply" continue to point to the same place and
>> find the data he needs?
>> Also, what about dfs.name.dir ? I guess I can simply point it to the
>> same place, right?
>> On  /home/hadoop/haddop_drive/hadoop/dfs I have data and name
>> directories. So I'm not sure if I should point the properties above to
>>  /home/hadoop/haddop_drive or to  /home/hadoop/haddop_drive/dfs.
>> Thanks,
>> JM
> --
> Harsh J

View raw message