Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4A83610161 for ; Thu, 16 Jan 2014 04:41:26 +0000 (UTC) Received: (qmail 35010 invoked by uid 500); 16 Jan 2014 04:41:22 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 34900 invoked by uid 500); 16 Jan 2014 04:41:20 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 34876 invoked by uid 99); 16 Jan 2014 04:41:20 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Jan 2014 04:41:20 +0000 Date: Thu, 16 Jan 2014 04:41:20 +0000 (UTC) From: "Ted Yu (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HBASE-10349) Table became unusable when master balanced its region after table was dropped MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-10349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-10349: --------------------------- Attachment: 10349-v1.txt Patch v1 adds a check that source server of the region hasn't changed. > Table became unusable when master balanced its region after table was dropped > ----------------------------------------------------------------------------- > > Key: HBASE-10349 > URL: https://issues.apache.org/jira/browse/HBASE-10349 > Project: HBase > Issue Type: Bug > Reporter: Ted Yu > Attachments: 10349-hadoop-hdfs-namenode-hor11n14.gq1.ygridcore.net.zip, 10349-output.log, 10349-v1.txt, hbase-hbase-master-hor15n05.gq1.ygridcore.net.log.tar.gz > > > 0.98 was used. > This was sequence of events: > create 'tablethree_mod' > snapshot 'tablethree_mod', 'snapshot_tablethree_mod' > disable 'tablethree_mod' > 2014-01-15 09:34:51,749 restore_snapshot 'snapshot_tablethree_mod' > 2014-01-15 09:35:07,210 enable 'tablethree_mod' > 2014-01-15 09:35:46,134 delete_snapshot 'snapshot_tablethree_mod' > 2014-01-15 09:41:42,210 disable 'tablethree_mod' > 2014-01-15 09:41:43,610 drop 'tablethree_mod' > create 'tablethree_mod' > For the last table creation request: > {code} > 2014-01-15 10:03:52,999|beaver.component.hbase|INFO| 'create 'tablethree_mod', > {NAME => 'f1', VERSIONS => 3} > , > {NAME => 'f2', VERSIONS => 3} > , > {NAME => 'f3', VERSIONS => 3} > ' > 2014-01-15 10:03:52,999|beaver.component.hbase|INFO| 'exists 'tablethree_mod'' > 2014-01-15 10:03:52,999|beaver.component.hbase|INFO| 'put 'tablethree_mod', '0', 'f1:q1', 'value-0', 10' > 2014-01-15 10:03:52,999|beaver.component.hbase|INFO| 'put 'tablethree_mod', '1', 'f1:q1', 'value-1', 20' > 2014-01-15 10:03:53,000|beaver.component.hbase|INFO| 'put 'tablethree_mod', '2', 'f2:q2', 'value-2', 30' > 2014-01-15 10:03:53,000|beaver.component.hbase|INFO| 'put 'tablethree_mod', '3', 'f3:q3', 'value-3', 40' > 2014-01-15 10:03:53,000|beaver.component.hbase|INFO| 'put 'tablethree_mod', '4', 'f3:q3', 'value-4', 50' > 2014-01-15 10:03:53,000|beaver.component.hbase|INFO|Done writing commands to file. Will execute them now. > 2014-01-15 10:03:53,000|beaver.machine|INFO|RUNNING: /usr/lib/hbase/bin/hbase shell /grid/0/tmp/hwqe/artifacts/tmp-471142 > 2014-01-15 10:03:55,878|beaver.machine|INFO|2014-01-15 10:03:55,878 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available > 2014-01-15 10:03:57,283|beaver.machine|INFO|2014-01-15 10:03:57,283 WARN [main] conf.Configuration: hbase-site.xml:an attempt to override final parameter: dfs.support.append; Ignoring. > 2014-01-15 10:03:57,669|beaver.machine|INFO|2014-01-15 10:03:57,669 WARN [main] conf.Configuration: hbase-site.xml:an attempt to override final parameter: dfs.support.append; Ignoring. > 2014-01-15 10:03:57,720|beaver.machine|INFO|2014-01-15 10:03:57,720 WARN [main] conf.Configuration: hbase-site.xml:an attempt to override final parameter: dfs.support.append; Ignoring. > 2014-01-15 10:03:57,997|beaver.machine|INFO| > 2014-01-15 10:03:57,997|beaver.machine|INFO|ERROR: Table already exists: tablethree_mod! > 2014-01-15 10:03:57,997|beaver.machine|INFO| > {code} > This was an intermittent issue after using Snapshots, a table is not properly dropped / and not able to properly re-create with the same name. And a HRegion is empty or null Error occurs. (When you try to drop the table it says it does not exist, and when you try to create the table it says that it does already exist). > {code} > 2014-01-15 10:04:02,462|beaver.machine|INFO|ERROR: HRegionInfo was null or empty in hbase:meta, row=keyvalues= > {tablethree_mod,,1389778226606.afc82d1ceabbaca36a504b83b65fc0c9./info:seqnumDuringOpen/1389778905355/Put/vlen=8/mvcc=0, tablethree_mod,,1389778226606.afc82d1ceabbaca36a504b83b65fc0c9./info:server/1389778905355/Put/vlen=32/mvcc=0, tablethree_mod,,1389778226606.afc82d1ceabbaca36a504b83b65fc0c9./info:serverstartcode/1389778905355/Put/vlen=8/mvcc=0} > {code} > Thanks to Huned who discovered this issue. -- This message was sent by Atlassian JIRA (v6.1.5#6160)