Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 6BC03200D1B for ; Thu, 12 Oct 2017 22:28:09 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 6A5F51609E4; Thu, 12 Oct 2017 20:28:09 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 95CC81609E8 for ; Thu, 12 Oct 2017 22:28:08 +0200 (CEST) Received: (qmail 12006 invoked by uid 500); 12 Oct 2017 20:28:07 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 11993 invoked by uid 99); 12 Oct 2017 20:28:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Oct 2017 20:28:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id E6A031A1704 for ; Thu, 12 Oct 2017 20:28:06 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id RHLGd-3s3iZZ for ; Thu, 12 Oct 2017 20:28:05 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id A5FB85FDDC for ; Thu, 12 Oct 2017 20:28:04 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id C8ED6E0B20 for ; Thu, 12 Oct 2017 20:28:03 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id A99F523F25 for ; Thu, 12 Oct 2017 20:28:02 +0000 (UTC) Date: Thu, 12 Oct 2017 20:28:02 +0000 (UTC) From: "Josh Elser (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-18998) processor.getRowsToLock() always assumes there is some row being locked MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 12 Oct 2017 20:28:09 -0000 [ https://issues.apache.org/jira/browse/HBASE-18998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202576#comment-16202576 ] Josh Elser commented on HBASE-18998: ------------------------------------ This is a bit strange looking (why, you might ask, is Phoenix calling {{mutateRowsWithLocks}} without specifying any rows to lock?). The Phoenix CP MetaDataEndpointImpl has already grabbed the necessary rows to lock that it needs. So here, when it's actually submitting the mutations that its built, it knows that it has already exclusively locked that row (necessary to prevent concurrent, conflicting DDL operations) and it provides no row locks. Pretty simple fix from our side that just looks a little weird. > processor.getRowsToLock() always assumes there is some row being locked > ----------------------------------------------------------------------- > > Key: HBASE-18998 > URL: https://issues.apache.org/jira/browse/HBASE-18998 > Project: HBase > Issue Type: Bug > Reporter: Ted Yu > > During testing, we observed the following exception: > {code} > 2017-10-12 02:52:26,683|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|1/1 DROP TABLE testTable; > 2017-10-12 02:52:30,320|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|17/10/12 02:52:30 WARN ipc.CoprocessorRpcChannel: Call failed on IOException > 2017-10-12 02:52:30,320|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|org.apache.hadoop.hbase.DoNotRetryIOException: org.apache.hadoop.hbase.DoNotRetryIOException: TESTTABLE: null > 2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:93) > 2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropTable(MetaDataEndpointImpl.java:1671) > 2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:14347) > 2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7849) > 2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1980) > 2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1962) > 2017-10-12 02:52:30,321|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32389) > 2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2150) > 2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112) > 2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187) > 2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167) > 2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|Caused by: java.util.NoSuchElementException > 2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at java.util.Collections$EmptyIterator.next(Collections.java:4189) > 2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.hadoop.hbase.regionserver.HRegion.processRowsWithLocks(HRegion.java:7137) > 2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.hadoop.hbase.regionserver.HRegion.mutateRowsWithLocks(HRegion.java:6980) > 2017-10-12 02:52:30,322|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateRowsWithLocks(MetaDataEndpointImpl.java:1966) > 2017-10-12 02:52:30,323|INFO|MainThread|machine.py:164 - run()||GUID=f4cd2a25-3040-41cc-b423-9ec7990048f4|at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropTable(MetaDataEndpointImpl.java:1650) > {code} > Here is code from branch-1.1 : > {code} > if (!mutations.isEmpty() && !walSyncSuccessful) { > LOG.warn("Wal sync failed. Roll back " + mutations.size() + > " memstore keyvalues for row(s):" + StringUtils.byteToHexString( > processor.getRowsToLock().iterator().next()) + "..."); > {code} > The assumption that processor.getRowsToLock().iterator() would always be non-empty was wrong. > In other branches, taking the iterator seems to have the same issue. > Thanks to [~elserj] who spotted this issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029)