Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 0A496200D33 for ; Wed, 8 Nov 2017 08:15:21 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 08C46160BE0; Wed, 8 Nov 2017 07:15:21 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 28E77160BDA for ; Wed, 8 Nov 2017 08:15:20 +0100 (CET) Received: (qmail 4631 invoked by uid 500); 8 Nov 2017 07:15:19 -0000 Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hbase.apache.org Delivered-To: mailing list dev@hbase.apache.org Received: (qmail 4620 invoked by uid 99); 8 Nov 2017 07:15:19 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Nov 2017 07:15:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 49A0ACFD93 for ; Wed, 8 Nov 2017 07:15:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id GQNcOs3NUwzL for ; Wed, 8 Nov 2017 07:15:17 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id E3FD264BD4 for ; Tue, 7 Nov 2017 21:14:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 12C68E0F65 for ; Tue, 7 Nov 2017 21:14:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 70AC4241AB for ; Tue, 7 Nov 2017 21:14:00 +0000 (UTC) Date: Tue, 7 Nov 2017 21:14:00 +0000 (UTC) From: "stack (JIRA)" To: dev@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HBASE-19205) Backport BASE-18441 ZookeeperWatcher#interruptedException should throw exception MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 08 Nov 2017 07:15:21 -0000 stack created HBASE-19205: ----------------------------- Summary: Backport BASE-18441 ZookeeperWatcher#interruptedException should throw exception Key: HBASE-19205 URL: https://issues.apache.org/jira/browse/HBASE-19205 Project: HBase Issue Type: Sub-task Components: Zookeeper Reporter: stack Assignee: stack Fix For: 1.2.7 In a branch-1.2 test run, I see a bunch of this in timed out test run: 709 2017-11-07 12:27:16,152 DEBUG [Time-limited test] zookeeper.ZooKeeperWatcher(760): hconnection-0xb1ad4ca0x0, quorum=localhost:50828, baseZNode=/hbase Received InterruptedException, doing nothing here 710 java.lang.InterruptedException Which is us swallowing interrupts out of zk. The suppression of the interrupt is messing us up.... We start to see this: {code} 5018 2017-11-07 12:30:18,276 WARN [Time-limited test] zookeeper.ZKUtil(378): master:367040x0, quorum=localhost:56068, baseZNode=/hbase Unable to set watcher on znode /hbase/backup-maste rs/ve0528.halxg.cloudera.com,36704,1510086438842 5019 java.lang.InterruptedException 5020 at java.lang.Object.wait(Native Method) 5021 at java.lang.Object.wait(Object.java:502) 5022 at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342) 5023 at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040) 5024 at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:220) 5025 at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:365) 5026 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:978) 5027 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 5028 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 5029 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 5030 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 5031 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 5032 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 5033 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 5034 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 5035 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 5036 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 5037 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) ... {code} Later I'm getting.... {code} 15578737 Caused by: java.lang.StackOverflowError 15578738 at java.security.AccessController.doPrivileged(Native Method) 15578739 at java.io.PrintWriter.(PrintWriter.java:116) 15578740 at java.io.PrintWriter.(PrintWriter.java:100) 15578741 at org.apache.log4j.DefaultThrowableRenderer.render(DefaultThrowableRenderer.java:58) 15578742 at org.apache.log4j.spi.ThrowableInformation.getThrowableStrRep(ThrowableInformation.java:87) 15578743 at org.apache.log4j.spi.LoggingEvent.getThrowableStrRep(LoggingEvent.java:413) 15578744 at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:313) 15578745 at org.apache.log4j.WriterAppender.append(WriterAppender.java:162) 15578746 at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251) 15578747 at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66) 15578748 at org.apache.log4j.Category.callAppenders(Category.java:206) 15578749 at org.apache.log4j.Category.forcedLog(Category.java:391) 15578750 at org.apache.log4j.Category.log(Category.java:856) 15578751 at org.apache.commons.logging.impl.Log4JLogger.warn(Log4JLogger.java:208) 15578752 at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:378) 15578753 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:978) 15578754 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 15578755 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 15578756 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 15578757 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 15578758 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 15578759 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) 15578760 at org.apache.hadoop.hbase.zookeeper.ZKUtil.createEphemeralNodeAndWatch(ZKUtil.java:980) .... {code} Let me backport the [~carp84] patch at least. -- This message was sent by Atlassian JIRA (v6.4.14#64029)