Return-Path: Delivered-To: apmail-hadoop-hbase-user-archive@minotaur.apache.org Received: (qmail 50471 invoked from network); 23 Apr 2009 09:53:58 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 23 Apr 2009 09:53:58 -0000 Received: (qmail 65261 invoked by uid 500); 23 Apr 2009 09:53:57 -0000 Delivered-To: apmail-hadoop-hbase-user-archive@hadoop.apache.org Received: (qmail 65222 invoked by uid 500); 23 Apr 2009 09:53:57 -0000 Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hbase-user@hadoop.apache.org Delivered-To: mailing list hbase-user@hadoop.apache.org Received: (qmail 65212 invoked by uid 99); 23 Apr 2009 09:53:57 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Apr 2009 09:53:57 +0000 X-ASF-Spam-Status: No, hits=-2.8 required=10.0 tests=RCVD_IN_DNSWL_MED,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [64.18.1.21] (HELO exprod6og108.obsmtp.com) (64.18.1.21) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Apr 2009 09:53:48 +0000 Received: from source ([192.150.8.22]) by exprod6ob108.postini.com ([64.18.5.12]) with SMTP ID DSNKSfA6lkQ1Y1Js58k4g196NawPbqt+3i+7@postini.com; Thu, 23 Apr 2009 02:53:28 PDT Received: from inner-relay-1.corp.adobe.com ([153.32.1.51]) by outbound-smtp-2.corp.adobe.com (8.12.10/8.12.10) with ESMTP id n3N9rOE0012044 for ; Thu, 23 Apr 2009 02:53:25 -0700 (PDT) Received: from nacas01.corp.adobe.com (nacas01.corp.adobe.com [10.8.189.99]) by inner-relay-1.corp.adobe.com (8.12.10/8.12.10) with ESMTP id n3N9qhis009318 for ; Thu, 23 Apr 2009 02:53:23 -0700 (PDT) Received: from eurcas01.eur.adobe.com (10.128.4.27) by nacas01.corp.adobe.com (10.8.189.99) with Microsoft SMTP Server (TLS) id 8.1.340.0; Thu, 23 Apr 2009 02:53:13 -0700 Received: from eurmbx01.eur.adobe.com ([10.128.4.32]) by eurcas01.eur.adobe.com ([192.168.168.110]) with mapi; Thu, 23 Apr 2009 10:53:12 +0100 From: Cosmin Lehene To: "hbase-user@hadoop.apache.org" Date: Thu, 23 Apr 2009 10:53:10 +0100 Subject: Recovering .META. table Thread-Topic: Recovering .META. table Thread-Index: AcnD+U82u73svPCq9ECAe+M2guCfoQ== Message-ID: Accept-Language: en-US Content-Language: en X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org Hi,=20 Is there any way to rebuild the .META. Table with information in HDFS? Long story short, the namenode ran out of disk and crashed and we couldn't have it load the edits.new file at startup (the edits.new file seemed corrupted and it failed to read some Long value) so we removed the edits.ne= w file. Apparently the edits file had 10 - 15 entries. Some edit data on -ROOT-, some edit data on .META., some data on a table compaction (comapction.dir) and some data on some hbase logs. Only last entry seem to be bogus, having almost a megabyte of trailing zeroes. After renaming edits.new HDFS started ok and HBase data is present. However when we started Hbase one table was missing entirely and one table was empty. The HBase master report shows Regions On FS: 94, but the report on region servers shows only 10 regions on all the region servers. Is there any way to cleanup hbase and rebuild the .META. table from HDFS? This would be ok even if it implies throwing up some data. If we would clean the edits.new file and restart HDFS or if we would manually apply the edits would that still make consistent changes to HBase data? Thanks, Cosmin=20