Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 05BD21060A for ; Mon, 23 Dec 2013 16:20:56 +0000 (UTC) Received: (qmail 2612 invoked by uid 500); 23 Dec 2013 16:20:49 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 2361 invoked by uid 500); 23 Dec 2013 16:20:48 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Delivered-To: moderator for user@hbase.apache.org Received: (qmail 25800 invoked by uid 99); 23 Dec 2013 13:52:19 -0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ibrahim.nelman@gmail.com designates 209.85.219.41 as permitted sender) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=3bs9d2CI7kMKWKCA7Pbi6W/YSuNIxf4YGACAzW1VPPs=; b=R6fRFEW3Lx/QGao+WdtRfXv/7qSbi7uV4Ou4AcJyijpZogS36yBDK2zA+i0D9/fmlL AEgxLW+uLYfC5PPAgQ6K+TECZ2foT7wDgvtL61EQACREXvaIWF79T/6ryQ/nZ6DreQ5B LaHxbZ/uCm0PljkVZ405zGlxOItt2JtmCZLL552Z7AYPkrgG1fRiwsAMW7xPhGaNmytu /j1hooY5dB/baz9AwuN4YRc0wYRcb1W105vy7Qxqer5fFP50R0OwX6JURXZEiecVNrrP +6HHqYo+yxd9Kggfpd5cskpnvgesArl7CwPOi8AfMfmXfOm8zscwRmjNEeyPPKONp8xO c2ew== MIME-Version: 1.0 X-Received: by 10.60.56.137 with SMTP id a9mr170328oeq.75.1387806711444; Mon, 23 Dec 2013 05:51:51 -0800 (PST) Date: Mon, 23 Dec 2013 21:51:51 +0800 Message-ID: Subject: HBase -ROOT- table have some hdfs blocks missing. (v.0.92) From: ibrahim nelman To: user@hbase.apache.org Content-Type: multipart/alternative; boundary=089e01294db20d7c2e04ee33ee47 X-Virus-Checked: Checked by ClamAV on apache.org --089e01294db20d7c2e04ee33ee47 Content-Type: text/plain; charset=ISO-8859-1 Hi All, I just shutdown my 10-nodes hbase cluster (0.92) and restart the machine. After restart, I can not start HBase because there is an hdfs file belongs to -ROOT- tables whose block is totally missing. Here is what hadoop fsck return : /hbase/-ROOT-/70236052/info/73f1a60fdb584d55a4ce38431c6221be 734 bytes, 1 block( s): /hbase/-ROOT-/70236052/info/73f1a60fdb584d55a4ce38431c6221be: CORRUPT blockpool BP-82693651-10.0.106.65-1339148449089 block blk_-284664395950138605 MISSING 1 blocks of total size 734 B 0. BP-82693651-10.0.106.65-1339148449089:blk_-284664395950138605_3976401 len=734 MISSING! I tried to repair using OfflineMetaRepair but the error is still there. Anyway to repair a -ROOT- table or generate it based on the data ? The other -ROOT- info files are still ok. I want to remove the missing link from the namenode, but worried that some meta can not be read afterward. I also found some .META. files corrupted, but will focus on fixing the -ROOT- first. Thank you. Regards, Nelman --089e01294db20d7c2e04ee33ee47--