hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Rodeghiero <paolo....@gmail.com>
Subject Re: Incompatible namespaceIDs after formatting namenode
Date Sun, 15 Jan 2012 23:01:41 GMT
Il 15/01/2012 09:45, gdan2000 ha scritto:
[...]
 > really not sure about removing data node directories.
 >
 > How is it possible that data will not be lost? I have to do it on all
 > datanodes...
 >
 > Please explain me how all this reformat tasks preserves user's data ?
 >

When you reformat the namenode, you are erasing and rebuilding what 
actually is the filesystem allocation table. As for traditional 
filesystems, you are not deleting the actual blocks.

Differently from the traditional scenario, the previously allocated 
space is not automatically reused, but datanodes that have blocks from 
the previous allocation will instead deny to link to the namenode.

I assume the main reason for this design choice is to preserve data from 
problems on name resolutions or configuration errors (i.e. the datanode 
trying to link to a wrong namenode)

Il 15/01/2012 21:16, Chen He ha scritto:
 > For short, here is a script that may be useful for your to remove hdfs
 > directory on DNs from your headnode.
[...]

You can also use slaves.sh in $HADOOP_HOME/bin to accomplish that:
it allows you to run a command on every slave node.

Cheers,
Paolo




Mime
View raw message