hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Loddengaard <a...@cloudera.com>
Subject Re: NameNode Startup Problem
Date Wed, 22 Apr 2009 18:03:27 GMT
Can you post your hadoop-site.xml?  Also, what prompted this problem?  Did
you bounce the cluster?

Alex

On Wed, Apr 22, 2009 at 8:16 AM, Tamir Kamara <tamirkamara@gmail.com> wrote:

> Hi,
>
> After a while working with hadoop I'm now faced with a situation where the
> namenode won't start up. I'm working with a patched up version of 0.19.1
> with ganglia patches (3422, 4675) and with 5269 which suppose to deal with
> killed_unclean task status and the massive "serious problem" lines in the
> JT
> logs.
> The latest NN logs are below.
>
> Can you help me figure out what is going on ?
>
> Thanks,
> Tamir
>
>
>
> 2009-04-22 18:12:36,966 INFO
> org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting NameNode
> STARTUP_MSG:   host = lb-emu-3/192.168.14.11
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = 0.19.2-dev
> STARTUP_MSG:   build =  -r ; compiled by 'tkamara' on Tue Apr 21 12:03:50
> IDT 2009
> ************************************************************/
> 2009-04-22 18:12:37,448 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
> Initializing RPC Metrics with hostName=NameNode, port=54310
> 2009-04-22 18:12:37,456 INFO
> org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
> lb-emu-3.israel.verisign.com/192.168.14.11:54310
> 2009-04-22<http://lb-emu-3.israel.verisign.com/192.168.14.11:54310%0A2009-04-22>18:12:37,467
INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
> Initializing JVM Metrics with processName=NameNode, sessionId=null
> 2009-04-22 18:12:37,474 INFO
> org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics:
> Initializing
> NameNodeMeterics using context object:org.apache.hadoop.metrics.spi.NullC
> ontext
> 2009-04-22 18:12:37,627 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=hadoop,hadoop
> 2009-04-22 18:12:37,628 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
> 2009-04-22 18:12:37,628 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
> isPermissionEnabled=true
> 2009-04-22 18:12:37,649 INFO
> org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics:
> Initializing FSNamesystemMetrics using context
> object:org.apache.hadoop.metrics.sp
> i.NullContext
> 2009-04-22 18:12:37,651 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered
> FSNamesystemStatusMBean
> 2009-04-22 18:12:37,814 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Number of files = 3427
> 2009-04-22 18:12:38,486 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Number of files under construction = 28
> 2009-04-22 18:12:38,511 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Image file of size 488333 loaded in 0 seconds.
> 2009-04-22 18:12:38,634 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Edits file /usr/local/hadoop-datastore/hadoop/dfs/name/current/edits of
> size
> 82110 edits # 477 loaded in
>  0 seconds.
> 2009-04-22 18:12:40,893 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Invalid opcode,
> reached
> end of edit log Number of transactions found 36635
> 2009-04-22 18:12:40,893 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Edits file /usr/local/hadoop-datastore/hadoop/dfs/name/current/edits.new of
> size 5229334 edits # 36635 l
> oaded in 2 seconds.
> 2009-04-22 18:12:41,024 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: saveLeases found path
>
> /tmp/temp623789763/tmp659456056/_temporary/_attempt_200904211331_0010_r_000002_0/part-00002
> but no matching entry in namespace.
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.saveFilesUnderConstruction(FSNamesystem.java:4608)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImage(FSImage.java:1010)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImage(FSImage.java:1031)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:88)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:309)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:288)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:163)
>        at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:208)
>        at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:194)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:859)
>        at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:868)
> 2009-04-22 18:12:41,038 INFO org.apache.hadoop.ipc.Server: Stopping server
> on 54310
> 2009-04-22 18:12:41,038 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> saveLeases found path
> /tmp/temp623789763/tmp659456056/_temporary/_attempt_20090
> 4211331_0010_r_000002_0/part-00002 but no matching entry in namespace.
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.saveFilesUnderConstruction(FSNamesystem.java:4608)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImage(FSImage.java:1010)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImage(FSImage.java:1031)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:88)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:309)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:288)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:163)
>        at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:208)
>        at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:194)
>        at
>
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:859)
>        at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:868)
>
> 2009-04-22 18:12:41,039 INFO
> org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at lb-emu-3/192.168.14.11
> ************************************************************/
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message