hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Kang <weliam.cl...@gmail.com>
Subject Re: Distributed hadoop setup 0 live datanode problem in cluster
Date Wed, 17 Mar 2010 07:18:51 GMT
Hi Jeff,
Here is the log from my namenode:


************************************************************/
2010-03-17 03:09:59,750 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = ubtserver01/127.0.0.1
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.2
STARTUP_MSG:   build =
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r
911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
2010-03-17 03:09:59,903 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
Initializing RPC Metrics with hostName=NameNode, port=9000
2010-03-17 03:09:59,909 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
ubtserver01/127.0.0.1:9000
2010-03-17 03:09:59,912 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=NameNode, sessionId=null
2010-03-17 03:09:59,914 INFO
org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics: Initializing
NameNodeMeterics using context
object:org.apache.hadoop.metrics.spi.NullContext
2010-03-17 03:09:59,979 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
fsOwner=cakang,cakang,adm,dialout,cdrom,plugdev,lpadmin,admin,sambashare
2010-03-17 03:09:59,980 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
2010-03-17 03:09:59,980 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
isPermissionEnabled=true
2010-03-17 03:09:59,995 INFO
org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics:
Initializing FSNamesystemMetrics using context
object:org.apache.hadoop.metrics.spi.NullContext
2010-03-17 03:09:59,998 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered
FSNamesystemStatusMBean
2010-03-17 03:10:00,042 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files = 1
2010-03-17 03:10:00,048 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files under construction = 0
2010-03-17 03:10:00,048 INFO org.apache.hadoop.hdfs.server.common.Storage:
Image file of size 96 loaded in 0 seconds.
2010-03-17 03:10:00,048 INFO org.apache.hadoop.hdfs.server.common.Storage:
Edits file /opt/hadoop/dfs/name/current/edits of size 4 edits # 0 loaded in
0 seconds.
2010-03-17 03:10:00,122 INFO org.apache.hadoop.hdfs.server.common.Storage:
Image file of size 96 saved in 0 seconds.
2010-03-17 03:10:00,391 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading
FSImage in 436 msecs
2010-03-17 03:10:00,393 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Total number of blocks
= 0
2010-03-17 03:10:00,393 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of invalid
blocks = 0
2010-03-17 03:10:00,393 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
under-replicated blocks = 0
2010-03-17 03:10:00,393 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of
 over-replicated blocks = 0
2010-03-17 03:10:00,393 INFO org.apache.hadoop.hdfs.StateChange: STATE*
Leaving safe mode after 0 secs.
2010-03-17 03:10:00,393 INFO org.apache.hadoop.hdfs.StateChange: STATE*
Network topology has 0 racks and 0 datanodes
2010-03-17 03:10:00,393 INFO org.apache.hadoop.hdfs.StateChange: STATE*
UnderReplicatedBlocks has 0 blocks
2010-03-17 03:10:05,572 INFO org.mortbay.log: Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog
2010-03-17 03:10:05,661 INFO org.apache.hadoop.http.HttpServer: Port
returned by webServer.getConnectors()[0].getLocalPort() before open() is -1.
Opening the listener on 50070
2010-03-17 03:10:05,663 INFO org.apache.hadoop.http.HttpServer:
listener.getLocalPort() returned 50070
webServer.getConnectors()[0].getLocalPort() returned 50070
2010-03-17 03:10:05,663 INFO org.apache.hadoop.http.HttpServer: Jetty bound
to port 50070
2010-03-17 03:10:05,663 INFO org.mortbay.log: jetty-6.1.14
2010-03-17 03:11:10,095 INFO org.mortbay.log: Started
SelectChannelConnector@0.0.0.0:50070
2010-03-17 03:11:10,095 INFO
org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at:
0.0.0.0:50070
2010-03-17 03:11:10,097 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2010-03-17 03:11:10,098 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 9000: starting
2010-03-17 03:11:10,110 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 9000: starting
2010-03-17 03:11:10,113 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 9000: starting
2010-03-17 03:11:10,114 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 1 on 9000: starting
2010-03-17 03:11:10,153 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 9000: starting
2010-03-17 03:11:10,153 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 9000: starting
2010-03-17 03:11:10,160 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 9000: starting
2010-03-17 03:11:10,160 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 9000: starting
2010-03-17 03:11:10,160 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 9000: starting
2010-03-17 03:11:10,161 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 9000: starting
2010-03-17 03:11:10,170 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 9000: starting
2010-03-17 03:15:51,270 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.registerDatanode: node registration from 127.0.0.1:50010 storage
DS-37429514-127.0.0.1-50010-1268798883208
2010-03-17 03:15:51,274 INFO org.apache.hadoop.net.NetworkTopology: Adding a
new node: /default-rack/127.0.0.1:50010

Thanks for the replies.
I am looking forward to hearing from you.


William

On Wed, Mar 17, 2010 at 3:14 AM, Jeff Zhang <zjffdu@gmail.com> wrote:

> Can you post your namenode's log ? It seems that your data node can not
> connect to the name node.
>
> On Wed, Mar 17, 2010 at 2:43 PM, William Kang <weliam.cloud@gmail.com
> >wrote:
>
> > Hi,
> > I just moved from pseudo distributed hadoop to a four machine full
> > distributed hadoop setup.
> >
> > But, after I start the dfs, there is no live node showing up. If I make
> > master a slave too, then the datanode in master machine will show up.
> >
> > I looked up all logs and found no errors. The only thing
> > looks suspicious  is the log in the datanode:
> >
> >
> > ************************************
> > 2010-03-17 02:39:04,003 INFO org.apache.hadoop.ipc.RPC: Server at
> > /xx.xx.xx.xx:9000 not available yet, Zzzzz...
> > 2010-03-17 02:39:06,064 INFO org.apache.hadoop.ipc.Client: Retrying
> connect
> > to server: /xx.xx.xx.xx:9000. Already tried 0 time(s).
> > 2010-03-17 02:39:07,076 INFO org.apache.hadoop.ipc.Client: Retrying
> connect
> > to server: /xx.xx.xx.xx:9000. Already tried 1 time(s).
> > 2010-03-17 02:39:08,081 INFO org.apache.hadoop.ipc.Client: Retrying
> connect
> > to server: /xx.xx.xx.xx:9000. Already tried 2 time(s).
> > 2010-03-17 02:39:09,098 INFO org.apache.hadoop.ipc.Client: Retrying
> connect
> > to server: /xx.xx.xx.xx6:9000. Already tried 3 time(s).
> > 2010-03-17 02:39:10,159 INFO org.apache.hadoop.ipc.Client: Retrying
> connect
> > to server: /xx.xx.xx.xx:9000. Already tried 4 time(s).
> > 2010-03-17 02:39:11,179 INFO org.apache.hadoop.ipc.Client: Retrying
> connect
> > to server: /xx.xx.xx.xx:9000. Already tried 5 time(s).
> > 2010-03-17 02:39:12,221 INFO org.apache.hadoop.ipc.Client: Retrying
> connect
> > to server: /xx.xx.xx.xx:9000. Already tried 6 time(s).
> > 2010-03-17 02:39:13,372 INFO org.apache.hadoop.ipc.Client: Retrying
> connect
> > to server: /xx.xx.xx.xx:9000. Already tried 7 time(s).
> > 2010-03-17 02:39:14,545 INFO org.apache.hadoop.ipc.Client: Retrying
> connect
> > to server: /xx.xx.xx.xx:9000. Already tried 8 time(s).
> > 2010-03-17 02:39:15,558 INFO org.apache.hadoop.ipc.Client: Retrying
> connect
> > to server: /xx.xx.xx.xx:9000. Already tried 9 time(s).
> > *************************************
> >
> > Does anybody know what might cause this problem?
> >
> > ssh among these machines are fine without password. The owner of hadoop
> > folder has been changed to the same hadoop user.
> >
> > Thanks!
> >
> >
> > William
> >
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message