curator-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mskonovalov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CURATOR-472) Test-only code started to be called in prod
Date Mon, 30 Jul 2018 01:04:00 GMT

    [ https://issues.apache.org/jira/browse/CURATOR-472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561320#comment-16561320
] 

mskonovalov edited comment on CURATOR-472 at 7/30/18 1:03 AM:
--------------------------------------------------------------

Here is dome thread dumps as well
{code:java}
"main@1" prio=5 tid=0x1 nid=NA waiting for monitor entry
java.lang.Thread.State: BLOCKED
waiting for main-EventThread@1594 to release lock on <0x716> (a org.apache.curator.ConnectionState)
at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:209)
at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:96)
at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:164)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:597)
at org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:268)
at org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:257)
at org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100)
at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:254)
at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:247)
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:206)
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:35)
at org.apache.curator.framework.imps.Test.testFastConnect(Test.java:21)
at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-1)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
at org.testng.TestRunner.privateRun(TestRunner.java:756)
at org.testng.TestRunner.run(TestRunner.java:610)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:387)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:382)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
at org.testng.SuiteRunner.run(SuiteRunner.java:289)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1293)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1218)
at org.testng.TestNG.runSuites(TestNG.java:1133)
at org.testng.TestNG.run(TestNG.java:1104)
at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

{code}
{code:java}
 "main-SendThread(192.0.2.0:2181)@1593" daemon prio=5 tid=0xf nid=NA sleeping java.lang.Thread.State:
TIMED_WAITING at java.lang.Thread.sleep(Thread.java:-1) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:336)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
{code}
{code:java}
"main-EventThread@1594" daemon prio=5 tid=0x10 nid=NA waiting
java.lang.Thread.State: WAITING
blocks Curator-ConnectionStateManager-0@1539
blocks main@1
at java.lang.Object.wait(Object.java:-1)
at java.lang.Thread.join(Thread.java:1252)
at org.apache.zookeeper.ZooKeeper.testableWaitForShutdown(ZooKeeper.java:2932)
at org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:1350)
at org.apache.curator.HandleHolder.internalClose(HandleHolder.java:158)
at org.apache.curator.HandleHolder.closeAndReset(HandleHolder.java:84)
at org.apache.curator.ConnectionState.reset(ConnectionState.java:203)
- locked <0x716> (a org.apache.curator.ConnectionState)
at org.apache.curator.ConnectionState.handleExpiredSession(ConnectionState.java:380)
at org.apache.curator.ConnectionState.checkState(ConnectionState.java:315)
at org.apache.curator.ConnectionState.process(ConnectionState.java:169)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:525)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:500)
{code}

{code}
"Curator-ConnectionStateManager-0@1539" daemon prio=5 tid=0xe nid=NA waiting for monitor entry
  java.lang.Thread.State: BLOCKED
	 waiting for main-EventThread@1594 to release lock on <0x716> (a org.apache.curator.ConnectionState)
	  at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:209)
	  at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:96)
	  at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:164)
	  at org.apache.curator.framework.state.ConnectionStateManager.checkSessionExpiration(ConnectionStateManager.java:307)
	  at org.apache.curator.framework.state.ConnectionStateManager.processEvents(ConnectionStateManager.java:283)
	  - locked <0x7cf> (a org.apache.curator.framework.state.ConnectionStateManager)
	  at org.apache.curator.framework.state.ConnectionStateManager.access$000(ConnectionStateManager.java:44)
	  at org.apache.curator.framework.state.ConnectionStateManager$1.call(ConnectionStateManager.java:120)
	  at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	  at java.util.concurrent.FutureTask.run(FutureTask.java:-1)
	  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	  at java.lang.Thread.run(Thread.java:748)
{code}


was (Author: mskonovalov):
Here is dome thread dumps as well


{code:java}
"main@1" prio=5 tid=0x1 nid=NA waiting for monitor entry
java.lang.Thread.State: BLOCKED
waiting for main-EventThread@1594 to release lock on <0x716> (a org.apache.curator.ConnectionState)
at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:209)
at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:96)
at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:164)
at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:597)
at org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:268)
at org.apache.curator.framework.imps.ExistsBuilderImpl$3.call(ExistsBuilderImpl.java:257)
at org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:64)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:100)
at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForegroundStandard(ExistsBuilderImpl.java:254)
at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:247)
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:206)
at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:35)
at org.apache.curator.framework.imps.Test.testFastConnect(Test.java:21)
at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-1)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
at org.testng.TestRunner.privateRun(TestRunner.java:756)
at org.testng.TestRunner.run(TestRunner.java:610)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:387)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:382)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
at org.testng.SuiteRunner.run(SuiteRunner.java:289)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1293)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1218)
at org.testng.TestNG.runSuites(TestNG.java:1133)
at org.testng.TestNG.run(TestNG.java:1104)
at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

{code}
{code:java}
 "main-SendThread(192.0.2.0:2181)@1593" daemon prio=5 tid=0xf nid=NA sleeping java.lang.Thread.State:
TIMED_WAITING at java.lang.Thread.sleep(Thread.java:-1) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:336)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
{code}
{code:java}
"main-EventThread@1594" daemon prio=5 tid=0x10 nid=NA waiting
java.lang.Thread.State: WAITING
blocks Curator-ConnectionStateManager-0@1539
blocks main@1
at java.lang.Object.wait(Object.java:-1)
at java.lang.Thread.join(Thread.java:1252)
at org.apache.zookeeper.ZooKeeper.testableWaitForShutdown(ZooKeeper.java:2932)
at org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:1350)
at org.apache.curator.HandleHolder.internalClose(HandleHolder.java:158)
at org.apache.curator.HandleHolder.closeAndReset(HandleHolder.java:84)
at org.apache.curator.ConnectionState.reset(ConnectionState.java:203)
- locked <0x716> (a org.apache.curator.ConnectionState)
at org.apache.curator.ConnectionState.handleExpiredSession(ConnectionState.java:380)
at org.apache.curator.ConnectionState.checkState(ConnectionState.java:315)
at org.apache.curator.ConnectionState.process(ConnectionState.java:169)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:525)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:500)
{code}

> Test-only code started to be called in prod
> -------------------------------------------
>
>                 Key: CURATOR-472
>                 URL: https://issues.apache.org/jira/browse/CURATOR-472
>             Project: Apache Curator
>          Issue Type: Bug
>          Components: Client
>            Reporter: mskonovalov
>            Assignee: Cameron McKenzie
>            Priority: Critical
>
> After commit {{fe2c7c4cd606c0cf4bc4fab15deedc0f4c33ea0e}} the method
>  
> {code:java}
> protected boolean testableWaitForShutdown(int wait)
>     throws InterruptedException
> {code}
> with the following comment
>  
>  
> {code:java}
> * Wait up to wait milliseconds for the underlying threads to shutdown.
> * THIS METHOD IS EXPECTED TO BE USED FOR TESTING ONLY!!!
> {code}
> started to be called from real method {{ZooKeeper.close()}}
> It prevents tests to finish.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message