From dev-return-9402-archive-asf-public=cust-asf.ponee.io@curator.apache.org Tue Nov 13 16:54:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 9A48D18062B for ; Tue, 13 Nov 2018 16:54:04 +0100 (CET) Received: (qmail 81816 invoked by uid 500); 13 Nov 2018 15:54:03 -0000 Mailing-List: contact dev-help@curator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@curator.apache.org Delivered-To: mailing list dev@curator.apache.org Received: (qmail 81797 invoked by uid 99); 13 Nov 2018 15:54:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Nov 2018 15:54:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 1201F19460A for ; Tue, 13 Nov 2018 15:54:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id afzSjkHlDOp1 for ; Tue, 13 Nov 2018 15:54:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 7A4A75F403 for ; Tue, 13 Nov 2018 15:54:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 013EEE092E for ; Tue, 13 Nov 2018 15:54:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id B93C725262 for ; Tue, 13 Nov 2018 15:54:00 +0000 (UTC) Date: Tue, 13 Nov 2018 15:54:00 +0000 (UTC) From: "Jordan Zimmerman (JIRA)" To: dev@curator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CURATOR-466) LeaderSelector gets in an inconsistent state when releasing resources. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CURATOR-466?page=3Dcom.atlassia= n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D166= 85401#comment-16685401 ]=20 Jordan Zimmerman commented on CURATOR-466: ------------------------------------------ Calling {{leaderSelector.close()=C2=A0}}will cause the=C2=A0LeaderSelectorL= istener to get called in a separate thread. However, you are immediately cl= osing the Curator handle after closing the leader selector therefore you're= getting the error you're seeing. So, this is expected. =C2=A0 {quote}do we really need to manually clean up on shutdown?=C2=A0 {quote} The reason for closing the leader selector is to immediately delete the ZNo= de so that any waiting leaders can become leader. If you don't close the le= ader selector you'd have to wait for the session to timeout.=C2=A0However, = in the case above closing the Curator handle (assuming it succeeds) will cl= ose the ZK handle and thus end the session. So, it's really up to you how t= o handle cleanup. But, closing the Curator Handle will quickly=C2=A0delete = any ephemeral ZNodes. > LeaderSelector gets in an inconsistent state when releasing resources. > ---------------------------------------------------------------------- > > Key: CURATOR-466 > URL: https://issues.apache.org/jira/browse/CURATOR-466 > Project: Apache Curator > Issue Type: Bug > Components: Recipes > Affects Versions: 4.0.1 > Reporter: Mikhail Pryakhin > Priority: Major > > I'm using the leader election recipe that works well until I encountered = application shutdown. > here is my example: > =C2=A0 > {code:java} > CuratorFramework framework =3D CuratorFrameworkFactory.builder() > .connectString("localhost:2181") > .retryPolicy(new RetryOneTime(100)) > .build(); > LeaderSelector leaderSelector =3D new LeaderSelector( > framework, > "/path", > new LeaderSelectorListener() { > volatile boolean stopped; > @Override > public void takeLeadership(CuratorFramework client) throws Except= ion { > System.out.println("I'm a new leader!"); > try { > while (!Thread.currentThread().isInterrupted() && !stoppe= d) { > TimeUnit.SECONDS.sleep(1); > } > } finally { > System.out.println("I'm not a leader anymore.."); > } > } > @Override > public void stateChanged(CuratorFramework client, ConnectionState= newState) { > if (client.getConnectionStateErrorPolicy().isErrorState(newSt= ate)) { > stopped =3D true; > } > } > } > ); > framework.start(); > leaderSelector.start(); > TimeUnit.SECONDS.sleep(5); > leaderSelector.close(); //(1) > framework.close(); //(2){code} > =C2=A0 > When I release resources by calling close method first on the LeaderSelec= tor instance and then on the CurtorFramework instance (lines 1 and 2) I alw= ays get the following exception: > =C2=A0 > {code:java} > java.lang.IllegalStateException: instance must be started before calling = this method > at org.apache.curator.shaded.com.google.common.base.Preconditions.checkSt= ate(Preconditions.java:444) ~[curator-client-4.0.1.jar:?] > at org.apache.curator.framework.imps.CuratorFrameworkImpl.delete(CuratorF= rameworkImpl.java:424) ~[curator-framework-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.locks.LockInternals.deleteOurPath= (LockInternals.java:347) ~[curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.locks.LockInternals.releaseLock(L= ockInternals.java:124) ~[curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.locks.InterProcessMutex.release(I= nterProcessMutex.java:154) ~[curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(Lead= erSelector.java:449) [curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(= LeaderSelector.java:466) [curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(= LeaderSelector.java:65) [curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(Lead= erSelector.java:246) [curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(Lead= erSelector.java:240) [curator-recipes-4.0.1.jar:4.0.1] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511= ) [?:1.8.0_141] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j= ava:1149) [?:1.8.0_141] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.= java:624) [?:1.8.0_141] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141] > {code} > =C2=A0 > The reason for the exception is that the non-blocking LeaderSelector.clos= e method delegates call to the internal executor service, which abruptly ca= ncels the running futures with the interptIfRunning flag set to true. Right= after this, the CuratorFramework close method is called. By the meantime, = the future being canceled executes the finally block where it calls methods= on the already closed CuratorFramework instance which leads to throwing an= exception. > I thought I can wait a bit until the LeaderSelector instance is closed, s= o I tried to delay for some time before closing the CuratorFramework instan= ce, but doing so leads to another exception: > {code:java} > ava.lang.InterruptedException: null > at java.lang.Object.wait(Native Method) ~[?:1.8.0_141] > at java.lang.Object.wait(Object.java:502) ~[?:1.8.0_141] > at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1409) ~[= zookeeper-3.4.12.jar:3.4.12--1] > at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:874) ~[zookeeper-= 3.4.12.jar:3.4.12--1] > at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuild= erImpl.java:274) ~[curator-framework-4.0.1.jar:4.0.1] > at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuild= erImpl.java:268) ~[curator-framework-4.0.1.jar:4.0.1] > at org.apache.curator.connection.StandardConnectionHandlingPolicy.callWit= hRetry(StandardConnectionHandlingPolicy.java:64) ~[curator-client-4.0.1.jar= :?] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100) ~[curat= or-client-4.0.1.jar:?] > at org.apache.curator.framework.imps.DeleteBuilderImpl.pathInForeground(D= eleteBuilderImpl.java:265) ~[curator-framework-4.0.1.jar:4.0.1] > at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuil= derImpl.java:249) ~[curator-framework-4.0.1.jar:4.0.1] > at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuil= derImpl.java:34) ~[curator-framework-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.locks.LockInternals.deleteOurPath= (LockInternals.java:347) ~[curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.locks.LockInternals.releaseLock(L= ockInternals.java:124) ~[curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.locks.InterProcessMutex.release(I= nterProcessMutex.java:154) ~[curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.leader.LeaderSelector.doWork(Lead= erSelector.java:449) [curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.leader.LeaderSelector.doWorkLoop(= LeaderSelector.java:466) [curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.leader.LeaderSelector.access$100(= LeaderSelector.java:65) [curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(Lead= erSelector.java:246) [curator-recipes-4.0.1.jar:4.0.1] > at org.apache.curator.framework.recipes.leader.LeaderSelector$2.call(Lead= erSelector.java:240) [curator-recipes-4.0.1.jar:4.0.1] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511= ) [?:1.8.0_141] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j= ava:1149) [?:1.8.0_141] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.= java:624) [?:1.8.0_141] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141] > {code} > At this time the exception is caused by the future being canceled with th= e interptIfRunning flag set to true in the LeaderSelector close method. > As the LeaderSelector implementation is based on the InterPorcessMutex th= at works with ephemeral nodes, do we really need to manually clean up on sh= utdown? As far as I know, the ephemeral nodes are deleted when the client d= isconnects. > =C2=A0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)