hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-820) NameNode startup fails if edit log terminates prematurely
Date Tue, 23 Oct 2007 22:40:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537164
] 

Konstantin Shvachko commented on HADOOP-820:
--------------------------------------------

You cannot rely on file mod times. It is not supported by all file systems, and time is not
expected to be synchronized across different 
(mounted) file systems. That is why the name-node creates a special file fstime, which contains
latest mod time for the image generated
centrally on the name-node once and the same for all image directories.
This issue is very distantly related to the issue you describe imo.

> NameNode startup fails if edit log terminates prematurely
> ---------------------------------------------------------
>
>                 Key: HADOOP-820
>                 URL: https://issues.apache.org/jira/browse/HADOOP-820
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>         Environment: ~50 node cluster
>            Reporter: Bryan Pendleton
>            Assignee: Raghu Angadi
>         Attachments: fixNameNodeStartup.patch
>
>
> I ran out of space on the device that stores the edit log, resulting in an edit log that
is truncated mid transaction.
> Ideally, the NameNode should start up, in SafeMode or the like, whenever this happens.
Right now, you get this stack trace:
> 2006-12-12 15:33:57,212 ERROR org.apache.hadoop.dfs.NameNode: java.io.EOFExcepti
> on
>         at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:310)
>         at org.apache.hadoop.io.UTF8.readFields(UTF8.java:104)
>         at org.apache.hadoop.dfs.FSEditLog.loadFSEdits(FSEditLog.java:227)
>         at org.apache.hadoop.dfs.FSImage.loadFSImage(FSImage.java:191)
>         at org.apache.hadoop.dfs.FSDirectory.loadFSImage(FSDirectory.java:320)
>         at org.apache.hadoop.dfs.FSNamesystem.<init>(FSNamesystem.java:226)
>         at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:146)
>         at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:138)
>         at org.apache.hadoop.dfs.NameNode.main(NameNode.java:589)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message