hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anandha L Ranganathan <analog.s...@gmail.com>
Subject Unable to restart standby Namenode.
Date Fri, 20 May 2016 19:00:45 GMT
Our standby namenode crashed and we couldn't restart the standby namenode.

I tried to manually restart it but still it is failed.  How do I recover
and restart the standby namenode.


2016-05-20 18:53:57,954 INFO  namenode.EditLogInputStream
(RedundantEditLogInputStream.java:nextOp(176)) - Fast-forwarding stream '
http://usw2stdpma01.glassdoor.local:8480/getJournal?jid=dfs-nameservices&segmentTxId=14726901&storageInfo=-60%3A761966699%3A0%3ACID-d16e0895-7c12-404e-9223-952d1b19ace0'
to transaction ID 13013207
2016-05-20 18:53:58,216 WARN  namenode.FSNamesystem
(FSNamesystem.java:loadFromDisk(750)) - Encountered exception loading
fsimage
java.io.IOException: There appears to be a gap in the edit log.  We
expected txid 13013207, but got txid 14726901.
    at
org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
    at
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:212)
    at
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140)
    at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:829)
    at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:684)
    at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281)
    at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1032)
    at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:748)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:538)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:597)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:764)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:748)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1441)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507)



2016-05-20 18:53:58,322 FATAL namenode.NameNode (NameNode.java:main(1512))
- Failed to start namenode.
java.io.IOException: There appears to be a gap in the edit log.  We
expected txid 13013207, but got txid 14726901.
    at
org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
    at
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:212)
    at
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:140)
    at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:829)
    at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:684)
    at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281)
    at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1032)
    at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:748)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:538)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:597)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:764)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:748)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1441)
    at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507)
2016-05-20 18:53:58,324 INFO  util.ExitUtil (ExitUtil.java:terminate(124))
- Exiting with status 1
2016-05-20 18:53:58,325 INFO  namenode.NameNode (StringUtils.java:run(659))
- SHUTDOWN_MSG:

Mime
View raw message