hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Shvachko <...@yahoo-inc.com>
Subject Re: DataNode often self-stopped
Date Tue, 03 Jun 2008 01:04:00 GMT
 > No , it is in different storage file.

What is in different storage file?
All data-nodes should have different configuration files, and each configuration file
should set a different storage directory property: "dfs.data.dir"
It is not a file, it is directory with all data-blocks.

 > the data-nodes self shut down ?

They do not SELF shut down. They are asked to shut down by the name-node
because there is a mismatch between previously registered ip address and storage ids.

According to the exception in your previous email, the data-node with ip address
192.168.10.7:50010 is trying to report a block on behalf of the node which is registered
under ip address 192.168.10.6:50010.
The funny thing is that their storage id DS-1812686469-192.168.10.5-50010-1211793342121
was created by yet another node 192.168.10.5:50010

I can only guess that VMWare somehow changes ip addresses on the fly.
Try to run data-nodes on different ports: 50011-14, then at least you will know which one
is talking now.

smallufo wrote:
> 2008/6/3 Konstantin Shvachko <shv@yahoo-inc.com>:
> 
> 
>>Is it possible that your different data-nodes point to the same storage
>>directory on
>>the hard drive? If so one of the data-nodes will be shut down.
> 
> 
> 
>>In general this is impossible because storage directories are locked once
>>one of the nodes
>>claims them under its authority. But I don't know whether this work in
>>VMWare environment.
>>
> 
> 
> No , it is in different storage file.
> 
> Is it because of network problem ?
> The VMWare-simulated network sometimes inter-ping time out because of host
> server's high load.
> Is it because of the network temporarily unavailable(for some seconds) ,
> results in the data-nodes self shut down ?
> 

Mime
View raw message