Return-Path: Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: (qmail 63979 invoked from network); 12 Apr 2011 19:22:46 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 12 Apr 2011 19:22:46 -0000 Received: (qmail 57607 invoked by uid 500); 12 Apr 2011 19:22:46 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 57581 invoked by uid 500); 12 Apr 2011 19:22:46 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 57573 invoked by uid 99); 12 Apr 2011 19:22:46 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Apr 2011 19:22:46 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,T_RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Apr 2011 19:22:43 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id C0A7F9ECEC for ; Tue, 12 Apr 2011 19:22:05 +0000 (UTC) Date: Tue, 12 Apr 2011 19:22:05 +0000 (UTC) From: "Aaron T. Myers (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: <1049343929.53173.1302636125785.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <28557936.153581295876025958.JavaMail.jira@thor> Subject: [jira] [Updated] (HDFS-1594) When the disk becomes full Namenode is getting shutdown and not able to recover MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HDFS-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1594: --------------------------------- Attachment: hdfs-1594.2.patch Thanks for the review, Cos. Not sure how I failed to notice that. Updated patch attached. > When the disk becomes full Namenode is getting shutdown and not able to recover > ------------------------------------------------------------------------------- > > Key: HDFS-1594 > URL: https://issues.apache.org/jira/browse/HDFS-1594 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node > Affects Versions: 0.21.0, 0.21.1, 0.22.0 > Environment: Linux linux124 2.6.27.19-5-default #1 SMP 2009-02-28 04:40:21 +0100 x86_64 x86_64 x86_64 GNU/Linux > Reporter: Devaraj K > Assignee: Aaron T. Myers > Fix For: 0.23.0 > > Attachments: HDFS-1594.patch, HDFS-1594.patch, HDFS-1594.patch, hadoop-root-namenode-linux124.log, hdfs-1594.0.patch, hdfs-1594.1.patch, hdfs-1594.2.patch > > > When the disk becomes full name node is shutting down and if we try to start after making the space available It is not starting and throwing the below exception. > {code:xml} > 2011-01-24 23:23:33,727 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed. > java.io.EOFException > at java.io.DataInputStream.readFully(DataInputStream.java:180) > at org.apache.hadoop.io.UTF8.readFields(UTF8.java:117) > at org.apache.hadoop.hdfs.server.namenode.FSImageSerialization.readString(FSImageSerialization.java:201) > at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:185) > at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:93) > at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:60) > at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSEdits(FSImage.java:1089) > at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:1041) > at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:487) > at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:149) > at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:306) > at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.(FSNamesystem.java:284) > at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:328) > at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:356) > at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:577) > at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:570) > at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1529) > at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1538) > 2011-01-24 23:23:33,729 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.EOFException > at java.io.DataInputStream.readFully(DataInputStream.java:180) > at org.apache.hadoop.io.UTF8.readFields(UTF8.java:117) > at org.apache.hadoop.hdfs.server.namenode.FSImageSerialization.readString(FSImageSerialization.java:201) > at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:185) > at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:93) > at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:60) > at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSEdits(FSImage.java:1089) > at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:1041) > at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:487) > at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:149) > at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:306) > at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.(FSNamesystem.java:284) > at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:328) > at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:356) > at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:577) > at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:570) > at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1529) > at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1538) > 2011-01-24 23:23:33,730 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG: > /************************************************************ > SHUTDOWN_MSG: Shutting down NameNode at linux124/10.18.52.124 > ************************************************************/ > {code} -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira