Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C028B1792E for ; Fri, 27 Mar 2015 00:11:58 +0000 (UTC) Received: (qmail 69159 invoked by uid 500); 27 Mar 2015 00:11:53 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 69117 invoked by uid 500); 27 Mar 2015 00:11:53 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 68822 invoked by uid 99); 27 Mar 2015 00:11:53 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Mar 2015 00:11:53 +0000 Date: Fri, 27 Mar 2015 00:11:53 +0000 (UTC) From: "Mikhail Antonov (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-9738) Delete table and loadbalancer interference MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383013#comment-14383013 ] Mikhail Antonov commented on HBASE-9738: ---------------------------------------- Seems to be related to HBASE-10349, yet that one deals with balancing of deleted table. > Delete table and loadbalancer interference > ------------------------------------------ > > Key: HBASE-9738 > URL: https://issues.apache.org/jira/browse/HBASE-9738 > Project: HBase > Issue Type: Bug > Reporter: Devaraj Das > Priority: Critical > Fix For: 2.0.0, 1.1.0 > > > I have noticed that when the balancer is computing a plan for region moves, and a delete table is issued, there is some interference. > 1. At time t1, user deleted the table. > 2. This led to the master updating the meta table to remove the line for the regioninfo for a region f2a9e2e9d70894c03f54ee5902bebee6. > {noformat} > 2013-10-04 08:42:52,495 INFO [MASTER_TABLE_OPERATIONS-hor15n05:60000-0] catalog.MetaEditor: Deleted [{ENCODED => f2a9e2e9d70894c03f54ee5902bebee6, NAME => 'usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6.', STARTKEY => '', ENDKEY => ''}] > {noformat} > 3. However around the same time, the balancer kicked in, and reassigned the region and made it online somewhere. It didn't check the fact (nor anyone else did) that the table was indeed deleted. > {noformat} > 2013-10-04 08:42:53,215 INFO [hor15n05.gq1.ygridcore.net,60000,1380869262259-BalancerChore] master.HMaster: balance hri=usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6., src=hor15n09.gq1.ygridcore.net,60020,1380869263722, dest=hor15n11.gq1.ygridcore.net,60020,1380869263682 > {noformat} > ..... > {noformat} > 2013-10-04 08:42:53,592 INFO [AM.ZK.Worker-pool2-t829] master.RegionStates: Onlined f2a9e2e9d70894c03f54ee5902bebee6 on hor15n11.gq1.ygridcore.net,60020,1380869263682 > {noformat} > 4. Henceforth, all the drop tables started giving warnings like > {noformat} > 2013-10-04 08:45:17,587 INFO [RpcServer.handler=8,port=60000] master.HMaster: Client=hrt_qa//68.142.246.151 delete usertable > 2013-10-04 08:45:17,631 DEBUG [RpcServer.handler=8,port=60000] lock.ZKInterProcessLockBase: Acquired a lock for /hbase/table-lock/usertable/write-master:600000000000000 > 2013-10-04 08:45:17,637 WARN [RpcServer.handler=8,port=60000] catalog.MetaReader: No serialized HRegionInfo in keyvalues={usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6./info:seqnumDuringOpen/1380876173509/Put/vlen=8/mvcc=0, usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6./info:server/1380876173509/Put/vlen=32/mvcc=0, usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6./info:serverstartcode/1380876173509/Put/vlen=8/mvcc=0} > {noformat} > 5. The create of the same table also fails since there is still state (reincarnated, maybe) about the table in the master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)