hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Shvachko <...@yahoo-inc.com>
Subject Re: DataNode often self-stopped
Date Mon, 02 Jun 2008 17:17:06 GMT
Is it possible that your different data-nodes point to the same storage directory on
the hard drive? If so one of the data-nodes will be shut down.
In general this is impossible because storage directories are locked once one of the nodes
claims them under its authority. But I don't know whether this work in VMWare environment.
--Konstantin

smallufo wrote:
> Hi
> 
> I am simulating a 4-DataNodes environment using VMWare.
> I found some data nodes often self-stopped after receiving a large file (or
> block).
> In fact , not so large , it is just smaller than 10MB.
> 
> This is the error messages :
> 
> 2008-05-27 16:40:54,727 INFO org.apache.hadoop.dfs.DataNode: Received block
> blk_3604066788791074317 of size 16777216 from /192.168.10.4
> 2008-05-27 16:40:54,727 INFO org.apache.hadoop.dfs.DataNode: PacketResponder
> 0 for block blk_3604066788791074317 terminating
> 2008-05-27 16:40:54,743 WARN org.apache.hadoop.dfs.DataNode: DataNode is
> shutting down: org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.dfs.UnregisteredDatanodeException: Data node
> 192.168.10.7:50010 is attempting to report storage ID
> DS-1812686469-192.168.10.5-50010-1211793342121. Node 192.168.10.6:50010 is
> expected to serve this storage.
>         at
> org.apache.hadoop.dfs.FSNamesystem.getDatanode(FSNamesystem.java:3594)
>         at
> org.apache.hadoop.dfs.FSNamesystem.blockReceived(FSNamesystem.java:3102)
>         at org.apache.hadoop.dfs.NameNode.blockReceived(NameNode.java:625)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 
>         at org.apache.hadoop.ipc.Client.call(Client.java:557)
>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>         at org.apache.hadoop.dfs.$Proxy4.blockReceived(Unknown Source)
>         at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:652)
>         at org.apache.hadoop.dfs.DataNode.run(DataNode.java:2667)
>         at java.lang.Thread.run(Thread.java:619)
> 
> 2008-05-27 16:40:54,745 INFO org.mortbay.util.ThreadedServer: Stopping
> Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=50075]
> 2008-05-27 16:40:54,753 INFO org.mortbay.http.SocketListener: Stopped
> SocketListener on 0.0.0.0:50075
> 2008-05-27 16:40:54,914 INFO org.mortbay.util.Container: Stopped
> HttpContext[/static,/static]
> 2008-05-27 16:40:54,950 INFO org.mortbay.util.Container: Stopped
> HttpContext[/logs,/logs]
> 2008-05-27 16:40:54,951 INFO org.mortbay.util.Container: Stopped
> org.mortbay.jetty.servlet.WebApplicationHandler@1950198
> 2008-05-27 16:40:55,044 INFO org.mortbay.util.Container: Stopped
> WebApplicationContext[/,/]
> 2008-05-27 16:40:55,044 INFO org.mortbay.util.Container: Stopped
> org.mortbay.jetty.Server@1c86be5
> 2008-05-27 16:40:55,044 INFO org.apache.hadoop.dfs.DataNode: Waiting for
> threa
> 2008-05-27 16:40:55,046 INFO org.apache.hadoop.dfs.DataNode:
> 192.168.10.7:5001
> 2008-05-27 16:40:55,284 INFO org.apache.hadoop.dfs.DataBlockScanner: Exiting
> D
> 2008-05-27 16:40:56,047 INFO org.apache.hadoop.dfs.DataNode: Waiting for
> threa
> 2008-05-27 16:40:56,050 INFO org.apache.hadoop.dfs.DataNode:
> 192.168.10.7:5001
> 2008-05-27 16:40:56,113 INFO org.apache.hadoop.dfs.DataNode: SHUTDOWN_MSG:
> 
> 
> What is the problem ?
> 

Mime
View raw message