Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0486DC23F for ; Tue, 29 May 2012 00:20:24 +0000 (UTC) Received: (qmail 18819 invoked by uid 500); 29 May 2012 00:20:23 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 18780 invoked by uid 500); 29 May 2012 00:20:23 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 18771 invoked by uid 99); 29 May 2012 00:20:23 -0000 Received: from issues-vm.apache.org (HELO issues-vm) (140.211.11.160) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 May 2012 00:20:23 +0000 Received: from isssues-vm.apache.org (localhost [127.0.0.1]) by issues-vm (Postfix) with ESMTP id 61B82140B94 for ; Tue, 29 May 2012 00:20:23 +0000 (UTC) Date: Tue, 29 May 2012 00:20:23 +0000 (UTC) From: "Kevin Odell (JIRA)" To: issues@hbase.apache.org Message-ID: <1965032581.9526.1338250823402.JavaMail.jiratomcat@issues-vm> In-Reply-To: <761242367.7990.1334876920277.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Commented] (HBASE-5837) hbase shell deleteall to .META. allows insertion of malformed rowkey. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-5837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284587#comment-13284587 ] Kevin Odell commented on HBASE-5837: ------------------------------------ I am working on a patch for this. > hbase shell deleteall to .META. allows insertion of malformed rowkey. > --------------------------------------------------------------------- > > Key: HBASE-5837 > URL: https://issues.apache.org/jira/browse/HBASE-5837 > Project: HBase > Issue Type: Bug > Components: master, shell > Affects Versions: 0.90.6 > Reporter: Jonathan Hsieh > > When using the hbase shell to manipulate meta entries, one is allowed to 'delete' malformed rows (entries with less than 2 ascii 44 ',' chars). When this happens HBase servers may go down and the cluster will not be restartable without manual intervention. > The delete results in a durable malformed rowkey in .META.'s memstore, .META.'s HLog, and eventually .META.'s HFiles. Subsequent scans to meta (such as when a HMaster starts) fail in the scanner because the comparator fails. In the case of an HMaster startup, it causes an abort that kills the HMaster process. > {code} > 12/04/18 22:07:34 FATAL master.HMaster: Unhandled exception. Starting shutdown. > org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.IllegalArgumentException: No 44 in , length=47, offset=54 > at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:990) > at org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:979) > at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1894) > at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1834) > at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570) > at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039) > Caused by: java.lang.IllegalArgumentException: No 44 in , length=47, offset=54 > at org.apache.hadoop.hbase.KeyValue.getRequiredDelimiterInReverse(KeyValue.java:1300) > at org.apache.hadoop.hbase.KeyValue$MetaKeyComparator.compareRows(KeyValue.java:1846) > at org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:130) > at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:257) > at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:114) > at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:2435) > at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:2391) > at org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:2408) > at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1870) > ... 6 more > at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257) > at $Proxy9.next(Unknown Source) > at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:264) > at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:237) > at org.apache.hadoop.hbase.catalog.MetaReader.fullScanOfResults(MetaReader.java:220) > at org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:1580) > at org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:221) > at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:422) > at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:295) > 12/04/18 22:07:34 INFO master.HMaster: Aborting > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira