hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From suresh srinivas <srini30...@gmail.com>
Subject Re: Data Nodes do not start
Date Wed, 09 Feb 2011 19:03:07 GMT
On Tue, Feb 8, 2011 at 11:05 PM, rahul patodi <patodirahul@gmail.com> wrote:

> I think you should copy the namespaceID of your master which is in
> name/current/VERSION file to all the slaves
>

This is a sure recipe for disaster. The VERSION file is a file system meta
data file not to be messed around with. At worst, this can cause loss of
entire file system data! Rahul please update your blog to reflect this.

Some background on namespace ID:
A namespace ID is created on the namenode when it is formatted. This is
propagated to datanodes when they register the first time with namenode.
>From then on, this ID is burnt into the datanodes.

A mismatch in namespace ID of datanode and namenode means:
# Datanode is pointing to a wrong namenode, perhaps in a different cluster
(config of datanode points to wrong namenode address).
# Namenode was running with a storage directory previously. It was changed
to some other storage directory with a different file system image.


Why does editing namespace ID is a bad idea?
Given that either namenode has loaded wrong namespace or datanode is
pointing to wrong namenode, messing around with namespaceID either on
namenode/datanode, results in datanode being able to register with the
namenode. When datanode sends block report, the blocks on the datanode do
not belong to the namespace loaded by the namenode, resulting in deletion of
all the blocks on the datanode.

Please find out if any of these problem exist in your setup and fix it.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message