geode-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bruce Schuchardt (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GEODE-3155) GMSJoinLeaveJUnitTest.testCoordinatorFindRequestSuccess failing in CI
Date Fri, 28 Jul 2017 16:38:00 GMT

    [ https://issues.apache.org/jira/browse/GEODE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105267#comment-16105267 ] 

Bruce Schuchardt commented on GEODE-3155:
-----------------------------------------

This is a new assertion in the test that [~hitesh.khamesra] added last month in 15a3b5a6eb789b01639a03c4318f5d3c10bd2158 when fixing GEODE-3052

> GMSJoinLeaveJUnitTest.testCoordinatorFindRequestSuccess failing in CI
> ---------------------------------------------------------------------
>
>                 Key: GEODE-3155
>                 URL: https://issues.apache.org/jira/browse/GEODE-3155
>             Project: Geode
>          Issue Type: Bug
>          Components: membership, messaging
>            Reporter: Galen O'Sullivan
>
> https://builds.apache.org/blue/organizations/jenkins/Geode-nightly/detail/Geode-nightly/880/tests
> I don't know what the problem is, needs debugging.
> Test output follows:
> {code}
> java.lang.AssertionError: Should be able to join
> Stacktrace
> java.lang.AssertionError: Should be able to join 
> 	at org.junit.Assert.fail(Assert.java:88)
> 	at org.junit.Assert.assertTrue(Assert.java:41)
> 	at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeaveJUnitTest.testCoordinatorFindRequestSuccess(GMSJoinLeaveJUnitTest.java:1218)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:114)
> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:57)
> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:66)
> 	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> 	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
> 	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
> 	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:109)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
> 	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:377)
> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
> 	at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:748)
> Standard Output
> [info 2017/06/28 11:05:53.704 UTC <Test worker> tid=0x13] received join request from asf902(31916)<ec>:8888
> [info 2017/06/28 11:05:53.728 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8891]
> old view is: null
> [info 2017/06/28 11:05:53.729 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888
> [info 2017/06/28 11:05:53.731 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:05:53.739 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v15>:8887
> [info 2017/06/28 11:05:53.740 UTC <Geode Membership View Creator> tid=0x19] View Creator thread is starting
> [info 2017/06/28 11:05:53.745 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:05:53.746 UTC <Test worker> tid=0x13] Membership received a request to remove asf902(31916)<ec>:8887 from asf902(31916)<ec>:8889 reason=testing
> [info 2017/06/28 11:05:53.752 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:05:53.754 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:05:53.754 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v12>:8887
> [info 2017/06/28 11:05:53.755 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec><v12>:8887]
> old view is: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec><v12>:8887, asf902(31916)<ec><v12>:8887]
> [info 2017/06/28 11:05:53.755 UTC <Geode Membership View Creator> tid=0x1c] View Creator thread is starting
> [info 2017/06/28 11:05:53.757 UTC <Geode Membership View Creator> tid=0x1c] preparing new view View[asf902(31916)<ec><v12>:8887|12] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec><v12>:8887, asf902(31916)<ec><v12>:8887]
> [info 2017/06/28 11:05:53.764 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:05:53.765 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8887{lead}, asf902(31916)<ec>:8887{lead}]
> old view is: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887, asf902(31916)<ec>:8887]
> [info 2017/06/28 11:05:53.765 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:05:53.766 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v10>:8887
> [info 2017/06/28 11:05:53.767 UTC <Geode Membership View Creator> tid=0x1e] View Creator thread is starting
> [info 2017/06/28 11:05:53.777 UTC <Thread-10> tid=0x20] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:05:53.796 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:05:53.796 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:05:53.796 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:05:53.797 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec><v0>:8887, asf902(31916)<ec>:8891]
> old view is: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> [info 2017/06/28 11:05:53.797 UTC <Geode Membership View Creator> tid=0x22] View Creator thread is starting
> [info 2017/06/28 11:05:53.925 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:05:53.925 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:05:53.926 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:05:53.927 UTC <Geode Membership View Creator> tid=0x27] View Creator thread is starting
> [info 2017/06/28 11:05:53.928 UTC <Geode Membership View Creator> tid=0x27] no recipients for new view aside from myself
> [info 2017/06/28 11:05:54.032 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:05:54.033 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887]
> old view is: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> [info 2017/06/28 11:05:54.034 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|3] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]  crashed: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8890]
> old view is: View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887]
> [warn 2017/06/28 11:05:54.034 UTC <Test worker> tid=0x13] total weight lost in this view change is 40 of 65.  Quorum has been lost!
> [info 2017/06/28 11:05:54.040 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:05:54.041 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887]
> old view is: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> [info 2017/06/28 11:05:54.041 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|3] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]  crashed: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8890]
> old view is: View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887]
> [warn 2017/06/28 11:05:54.041 UTC <Test worker> tid=0x13] total weight lost in this view change is 40 of 65.  Quorum has been lost!
> [info 2017/06/28 11:05:54.046 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:05:54.046 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:05:54.046 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:05:54.047 UTC <Geode Membership View Creator> tid=0x2d] View Creator thread is starting
> [info 2017/06/28 11:05:54.049 UTC <Geode Membership View Creator> tid=0x2d] no recipients for new view aside from myself
> [info 2017/06/28 11:05:54.547 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> old view is: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> Current viewid 1
> [info 2017/06/28 11:05:54.551 UTC <Test worker> tid=0x13] received join request from asf902(31916)<ec>:8890
> [info 2017/06/28 11:05:54.853 UTC <Geode Membership View Creator> tid=0x2d] View Creator is processing 1 requests for the next membership view
> [info 2017/06/28 11:05:54.854 UTC <Geode Membership View Creator> tid=0x2d] preparing new view View[asf902(31916)<ec><v0>:8887|2] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890]
> Empty sleeps 1 stopping: false
> Empty sleeps 2 stopping: false
> Empty sleeps 3 stopping: false
> Empty sleeps 4 stopping: false
> Empty sleeps 5 stopping: false
> [info 2017/06/28 11:05:59.856 UTC <Geode Membership View Creator> tid=0x2d] finished waiting for responses to view preparation
> [warn 2017/06/28 11:05:59.857 UTC <Geode Membership View Creator> tid=0x2d] these members failed to respond to the view change: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8888, asf902(31916)<ec><v2>:8890]
> [info 2017/06/28 11:05:59.862 UTC <Geode Membership View Creator> tid=0x2d] adding these unresponsive members to the crash-set for the next view: [asf902(31916)<ec>:8888]
> [info 2017/06/28 11:05:59.863 UTC <Geode Membership View Creator> tid=0x2d]   asf902(31916)<ec>:8888 had a weight of 10
> [info 2017/06/28 11:05:59.863 UTC <Geode Membership View Creator> tid=0x2d] preparing new view View[asf902(31916)<ec><v0>:8887|3] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890]  crashed: [asf902(31916)<ec>:8888]
> Empty sleeps 6 stopping: false
> Empty sleeps 7 stopping: false
> Empty sleeps 8 stopping: false
> Empty sleeps 9 stopping: false
> Empty sleeps 10 stopping: false
> [info 2017/06/28 11:06:04.865 UTC <Geode Membership View Creator> tid=0x2d] finished waiting for responses to view preparation
> [warn 2017/06/28 11:06:04.865 UTC <Geode Membership View Creator> tid=0x2d] these members failed to respond to the view change: [asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890]
> [info 2017/06/28 11:06:04.905 UTC <Geode Membership View Creator> tid=0x2d] received new view: View[asf902(31916)<ec><v0>:8887|3] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890]  crashed: [asf902(31916)<ec>:8888]
> old view is: View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> [info 2017/06/28 11:06:04.906 UTC <Geode Membership View Creator> tid=0x2d]   asf902(31916)<ec>:8888 had a weight of 10
> [info 2017/06/28 11:06:04.906 UTC <Geode Membership View Creator> tid=0x2d] sending new view View[asf902(31916)<ec><v0>:8887|3] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890]  crashed: [asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:04.907 UTC <Geode Membership View Creator> tid=0x2d] View Creator is processing 1 requests for the next membership view
> Empty sleeps 11 stopping: false
> [info 2017/06/28 11:06:05.570 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v1>:8887|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8891, asf902(31916)<ec><v1>:8887]
> old view is: null
> [info 2017/06/28 11:06:05.571 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec><v1>:8887
> [info 2017/06/28 11:06:05.572 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v1>:8887
> [info 2017/06/28 11:06:05.574 UTC <Geode Membership View Creator> tid=0x35] View Creator thread is starting
> [info 2017/06/28 11:06:05.586 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:06:05.590 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887, asf902(31916)<ec>:8891]
> old view is: null
> [info 2017/06/28 11:06:05.591 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8889 for asf902(31916)<ec>:8889
> [info 2017/06/28 11:06:05.591 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8890 for asf902(31916)<ec>:8890
> [info 2017/06/28 11:06:05.591 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|3] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887, asf902(31916)<ec>:8889, asf902(31916)<ec>:8891]
> old view is: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887, asf902(31916)<ec>:8891]
> [info 2017/06/28 11:06:05.592 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888
> [info 2017/06/28 11:06:05.592 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:05.593 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v15>:8887
> [info 2017/06/28 11:06:05.593 UTC <Geode Membership View Creator> tid=0x39] View Creator thread is starting
> [info 2017/06/28 11:06:05.597 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887, asf902(31916)<ec>:8891]
> old view is: null
> [info 2017/06/28 11:06:05.597 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8889 for asf902(31916)<ec>:8889
> [info 2017/06/28 11:06:05.597 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8890 for asf902(31916)<ec>:8890
> [info 2017/06/28 11:06:05.598 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888
> [info 2017/06/28 11:06:05.598 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:05.599 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v8>:8887
> [info 2017/06/28 11:06:05.599 UTC <Geode Membership View Creator> tid=0x3b] View Creator thread is starting
> [info 2017/06/28 11:06:05.606 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, asf902(31916)<ec>:8888]
> old view is: null
> [info 2017/06/28 11:06:05.607 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8889 for asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:05.607 UTC <Test worker> tid=0x13] I am being told to leave the distributed system by asf902(31916)<ec>:8889
> [info 2017/06/28 11:06:05.610 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:05.611 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:06:05.611 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:06:05.612 UTC <Test worker> tid=0x13] received join request from asf902(31916)<ec>:8888
> [info 2017/06/28 11:06:05.612 UTC <Geode Membership View Creator> tid=0x3e] View Creator thread is starting
> [info 2017/06/28 11:06:05.617 UTC <Geode Membership View Creator> tid=0x3e] no recipients for new view aside from myself
> [info 2017/06/28 11:06:05.918 UTC <Geode Membership View Creator> tid=0x3e] View Creator is processing 1 requests for the next membership view
> [info 2017/06/28 11:06:05.919 UTC <Geode Membership View Creator> tid=0x3e] preparing new view View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec><v1>:8888]
> [info 2017/06/28 11:06:05.925 UTC <Geode Membership View Creator> tid=0x3e] View Creator thread interrupted
> [info 2017/06/28 11:06:05.931 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|3] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8887, asf902(31916)<ec>:8891]
> old view is: null
> [info 2017/06/28 11:06:05.934 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, asf902(31916)<ec>:8888]
> old view is: null
> [info 2017/06/28 11:06:05.935 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:05.936 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v6>:8887
> [info 2017/06/28 11:06:05.936 UTC <Test worker> tid=0x13] Membership received a request to remove asf902(31916)<ec>:8888 from asf902(31916)<ec>:8888 reason=testing
> [info 2017/06/28 11:06:05.937 UTC <Geode Membership View Creator> tid=0x43] View Creator thread is starting
> [info 2017/06/28 11:06:05.937 UTC <Test worker> tid=0x13] Membership received a request to remove asf902(31916)<ec>:8888 from asf902(31916)<ec>:8888 reason=testing
> [info 2017/06/28 11:06:05.939 UTC <Geode Membership View Creator> tid=0x43] preparing new view View[asf902(31916)<ec><v6>:8887|6] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> [info 2017/06/28 11:06:10.941 UTC <Geode Membership View Creator> tid=0x43] finished waiting for responses to view preparation
> [warn 2017/06/28 11:06:10.941 UTC <Geode Membership View Creator> tid=0x43] these members failed to respond to the view change: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:10.942 UTC <Geode View Creator verification thread 1> tid=0x44] checking state of member asf902(31916)<ec>:8889
> [info 2017/06/28 11:06:10.943 UTC <Geode View Creator verification thread 1> tid=0x44] member asf902(31916)<ec>:8889 failed availability check
> [info 2017/06/28 11:06:15.917 UTC <Geode Membership View Creator> tid=0x43] adding these unresponsive members to the crash-set for the next view: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:15.918 UTC <Geode Membership View Creator> tid=0x43] preparing new view View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}]  crashed: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:15.918 UTC <Geode Membership View Creator> tid=0x43] finished waiting for responses to view preparation
> [info 2017/06/28 11:06:15.918 UTC <Geode Membership View Creator> tid=0x43] received new view: View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}]  crashed: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8888]
> old view is: View[asf902(31916)<ec><v6>:8887|1] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> [info 2017/06/28 11:06:15.918 UTC <Geode Membership View Creator> tid=0x43]   asf902(31916)<ec>:8889 had a weight of 10
> [info 2017/06/28 11:06:15.918 UTC <Geode Membership View Creator> tid=0x43]   asf902(31916)<ec>:8888 had a weight of 10
> [warn 2017/06/28 11:06:15.919 UTC <Geode Membership View Creator> tid=0x43] total weight lost in this view change is 20 of 35.  Quorum has been lost!
> [info 2017/06/28 11:06:15.920 UTC <Geode Membership View Creator> tid=0x43] sending new view View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}]  crashed: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:16.221 UTC <Geode Membership View Creator> tid=0x43] View Creator is processing 1 requests for the next membership view
> [info 2017/06/28 11:06:16.942 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:16.942 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:06:16.942 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:06:16.942 UTC <Geode Membership View Creator> tid=0x46] View Creator thread is starting
> [info 2017/06/28 11:06:16.943 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec><v0>:8887, asf902(31916)<ec>:8891]
> old view is: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> [info 2017/06/28 11:06:16.945 UTC <Geode Membership View Creator> tid=0x46] no recipients for new view aside from myself
> [info 2017/06/28 11:06:17.245 UTC <Geode Membership View Creator> tid=0x46] View Creator is processing 3 requests for the next membership view
> [info 2017/06/28 11:06:17.247 UTC <Geode Membership View Creator> tid=0x46]   asf902(31916)<ec>:8889 had a weight of 10
> [info 2017/06/28 11:06:17.248 UTC <Geode Membership View Creator> tid=0x46]   asf902(31916)<ec>:8891 had a weight of 10
> [info 2017/06/28 11:06:17.248 UTC <Geode Membership View Creator> tid=0x46]   asf902(31916)<ec>:8890 had a weight of 10
> [warn 2017/06/28 11:06:17.248 UTC <Geode Membership View Creator> tid=0x46] total weight lost in this view change is 30 of 55.  Quorum has been lost!
> [info 2017/06/28 11:06:17.250 UTC <Geode Membership View Creator> tid=0x46] View Creator thread interrupted
> [info 2017/06/28 11:06:17.254 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:06:17.260 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:06:17.260 UTC <Test worker> tid=0x13] Membership ignoring removal request for asf902(31916)<ec>:8889 from non-member asf902(31916)<ec>:9000
> [info 2017/06/28 11:06:17.264 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:17.264 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:06:17.265 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [fatal 2017/06/28 11:06:17.265 UTC <Test worker> tid=0x13] This member is no longer in the membership view.  My ID is asf902(31916)<ec><v0>:8887 and the new view is View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec>:8891]
> [info 2017/06/28 11:06:17.265 UTC <Geode Membership View Creator> tid=0x4a] View Creator thread is starting
> [info 2017/06/28 11:06:18.267 UTC <Geode Membership View Creator> tid=0x4a] no recipients for new view aside from myself
> [info 2017/06/28 11:06:18.271 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:06:18.275 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:06:18.277 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8889 for asf902(31916)<ec>:8889
> [info 2017/06/28 11:06:18.277 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:18.278 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v11>:8887
> [info 2017/06/28 11:06:18.279 UTC <Geode Membership View Creator> tid=0x4d] View Creator thread is starting
> [info 2017/06/28 11:06:18.296 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:18.297 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:06:18.298 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:06:18.299 UTC <Geode Membership View Creator> tid=0x4f] View Creator thread is starting
> [info 2017/06/28 11:06:19.300 UTC <Geode Membership View Creator> tid=0x4f] no recipients for new view aside from myself
> [info 2017/06/28 11:06:19.304 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:19.305 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:06:19.305 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:06:19.306 UTC <Geode Membership View Creator> tid=0x51] View Creator thread is starting
> [info 2017/06/28 11:06:19.310 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:19.311 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:06:19.311 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:06:19.311 UTC <Geode Membership View Creator> tid=0x53] View Creator thread is starting
> [info 2017/06/28 11:06:19.314 UTC <Geode Membership View Creator> tid=0x53] no recipients for new view aside from myself
> [info 2017/06/28 11:06:19.812 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> old view is: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> Current viewid 1
> [info 2017/06/28 11:06:19.812 UTC <Test worker> tid=0x13] received join request from asf902(31916)<ec>:8890
> [info 2017/06/28 11:06:20.113 UTC <Geode Membership View Creator> tid=0x53] View Creator is processing 1 requests for the next membership view
> [info 2017/06/28 11:06:20.114 UTC <Geode Membership View Creator> tid=0x53] preparing new view View[asf902(31916)<ec><v0>:8887|2] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890]
> Empty sleeps 1 stopping: false
> Empty sleeps 2 stopping: false
> Empty sleeps 3 stopping: false
> Empty sleeps 4 stopping: false
> Empty sleeps 5 stopping: false
> [info 2017/06/28 11:06:25.116 UTC <Geode Membership View Creator> tid=0x53] finished waiting for responses to view preparation
> [warn 2017/06/28 11:06:25.117 UTC <Geode Membership View Creator> tid=0x53] these members failed to respond to the view change: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8888, asf902(31916)<ec><v2>:8890]
> [info 2017/06/28 11:06:25.118 UTC <Geode View Creator verification thread 2> tid=0x55] received leave request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888
> [info 2017/06/28 11:06:25.121 UTC <Geode Membership View Creator> tid=0x53] received new view: View[asf902(31916)<ec><v0>:8887|2] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890]
> old view is: View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> [info 2017/06/28 11:06:25.123 UTC <Geode Membership View Creator> tid=0x53] sending new view View[asf902(31916)<ec><v0>:8887|2] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890]
> [info 2017/06/28 11:06:25.123 UTC <Geode Membership View Creator> tid=0x53] View Creator is processing 1 requests for the next membership view
> [info 2017/06/28 11:06:25.124 UTC <Geode Membership View Creator> tid=0x53] preparing new view View[asf902(31916)<ec><v0>:8887|3] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec><v2>:8890]  shutdown: [asf902(31916)<ec>:8888]
> Empty sleeps 6 stopping: false
> [info 2017/06/28 11:06:25.814 UTC <Geode Membership View Creator> tid=0x53] View Creator thread interrupted
> [info 2017/06/28 11:06:25.823 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, asf902(31916)<ec>:8888]
> old view is: null
> [info 2017/06/28 11:06:25.825 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:25.826 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v6>:8887
> [info 2017/06/28 11:06:25.828 UTC <Geode Membership View Creator> tid=0x58] View Creator thread is starting
> [info 2017/06/28 11:06:25.830 UTC <Geode Membership View Creator> tid=0x58] preparing new view View[asf902(31916)<ec><v6>:8887|6] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:25.930 UTC <Geode Membership View Creator> tid=0x58] finished waiting for responses to view preparation
> [warn 2017/06/28 11:06:25.931 UTC <Geode Membership View Creator> tid=0x58] received a conflicting membership view from asf902(31916)<ec>:8888 during preparation: View[asf902(31916:locator)<ec>:8892|11] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916:locator)<ec>:8892]
> [info 2017/06/28 11:06:25.931 UTC <Geode Membership View Creator> tid=0x58] View preparation interrupted - a locator is taking over as membership coordinator in this view: View[asf902(31916:locator)<ec>:8892|11] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916:locator)<ec>:8892]
> [info 2017/06/28 11:06:26.138 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:26.138 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:06:26.138 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:06:26.139 UTC <Geode Membership View Creator> tid=0x60] View Creator thread is starting
> [info 2017/06/28 11:06:26.149 UTC <Geode Membership View Creator> tid=0x60] no recipients for new view aside from myself
> [info 2017/06/28 11:06:26.241 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890]
> old view is: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> [info 2017/06/28 11:06:26.242 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8889 for asf902(31916)<ec>:8889
> [info 2017/06/28 11:06:31.248 UTC <Geode Membership View Creator> tid=0x60] View Creator is processing 1 requests for the next membership view
> [info 2017/06/28 11:06:31.249 UTC <Geode Membership View Creator> tid=0x60] preparing new view View[asf902(31916)<ec><v0>:8887|2] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8890]  shutdown: [asf902(31916)<ec>:8889]
> [info 2017/06/28 11:06:36.251 UTC <Geode Membership View Creator> tid=0x60] finished waiting for responses to view preparation
> [warn 2017/06/28 11:06:36.251 UTC <Geode Membership View Creator> tid=0x60] these members failed to respond to the view change: [asf902(31916)<ec>:8890]
> [info 2017/06/28 11:06:36.251 UTC <Geode View Creator verification thread 1> tid=0x65] checking state of member asf902(31916)<ec>:8890
> [info 2017/06/28 11:06:36.252 UTC <Geode View Creator verification thread 1> tid=0x65] member asf902(31916)<ec>:8890 passed availability check
> [info 2017/06/28 11:06:36.253 UTC <Geode Membership View Creator> tid=0x60] received new view: View[asf902(31916)<ec><v0>:8887|2] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8890]  shutdown: [asf902(31916)<ec>:8889]
> old view is: View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890]
> [info 2017/06/28 11:06:36.254 UTC <Geode Membership View Creator> tid=0x60] sending new view View[asf902(31916)<ec><v0>:8887|2] members: [asf902(31916)<ec><v0>:8887{lead}, asf902(31916)<ec>:8890]  shutdown: [asf902(31916)<ec>:8889]
> [info 2017/06/28 11:06:36.280 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, asf902(31916)<ec>:8888]
> old view is: null
> [info 2017/06/28 11:06:36.281 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:36.281 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v6>:8887
> [info 2017/06/28 11:06:36.281 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888
> [info 2017/06/28 11:06:36.281 UTC <Geode Membership View Creator> tid=0x67] View Creator thread is starting
> [info 2017/06/28 11:06:36.281 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888
> [info 2017/06/28 11:06:36.283 UTC <Geode Membership View Creator> tid=0x67] preparing new view View[asf902(31916)<ec><v6>:8887|6] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:36.284 UTC <Geode Membership View Creator> tid=0x67] finished waiting for responses to view preparation
> [info 2017/06/28 11:06:36.285 UTC <Geode Membership View Creator> tid=0x67] received new view: View[asf902(31916)<ec><v6>:8887|6] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888]
> old view is: View[asf902(31916)<ec><v6>:8887|1] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:36.285 UTC <Geode Membership View Creator> tid=0x67] sending new view View[asf902(31916)<ec><v6>:8887|6] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:36.586 UTC <Geode Membership View Creator> tid=0x67] View Creator is processing 2 requests for the next membership view
> [info 2017/06/28 11:06:36.586 UTC <Geode Membership View Creator> tid=0x67] received new view: View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}]  shutdown: [asf902(31916)<ec>:8888]
> old view is: View[asf902(31916)<ec><v6>:8887|6] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:36.587 UTC <Geode Membership View Creator> tid=0x67] no recipients for new view aside from myself
> [info 2017/06/28 11:06:37.286 UTC <Test worker> tid=0x13] received join request from asf902(31916)<ec>:8700(version:GFE 5.6)
> [warn 2017/06/28 11:06:37.286 UTC <Test worker> tid=0x13] detected an attempt to start a peer using an older version of the product asf902(31916)<ec>:8700(version:GFE 5.6)
> [info 2017/06/28 11:06:37.290 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:06:37.290 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec><v0>:8889 for asf902(31916)<ec><v0>:8889
> [info 2017/06/28 11:06:37.293 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:06:37.294 UTC <Test worker> tid=0x13] Membership received a request to remove asf902(31916)<ec>:8888 from asf902(31916)<ec>:8888 reason=testing
> [info 2017/06/28 11:06:37.294 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:37.295 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v13>:8887
> [info 2017/06/28 11:06:37.295 UTC <Geode Membership View Creator> tid=0x6b] View Creator thread is starting
> [info 2017/06/28 11:06:37.297 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:37.298 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:06:37.298 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:06:37.299 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8889, asf902(31916)<ec>:8890, asf902(31916)<ec><v0>:8887, asf902(31916)<ec>:8891]
> old view is: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> [info 2017/06/28 11:06:37.299 UTC <Geode Membership View Creator> tid=0x6d] View Creator thread is starting
> [info 2017/06/28 11:06:37.301 UTC <Test worker> tid=0x13] received join request from asf902(31916)<ec>:8888
> [info 2017/06/28 11:06:37.304 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:37.304 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec><v0>:8887|0] members: [asf902(31916)<ec><v0>:8887{lead}]
> old view is: null
> [info 2017/06/28 11:06:37.304 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v0>:8887
> [info 2017/06/28 11:06:37.307 UTC <Geode Membership View Creator> tid=0x70] View Creator thread is starting
> [info 2017/06/28 11:06:37.309 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:06:37.310 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8887{lead}, asf902(31916)<ec>:8890]
> old view is: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> [info 2017/06/28 11:06:37.310 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:37.310 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v14>:8887
> [info 2017/06/28 11:06:37.311 UTC <Geode Membership View Creator> tid=0x72] View Creator thread is starting
> [info 2017/06/28 11:06:37.312 UTC <Geode Membership View Creator> tid=0x72] preparing new view View[asf902(31916)<ec><v14>:8887|14] members: [asf902(31916)<ec><v14>:8887{lead}, asf902(31916)<ec>:8890]
> [info 2017/06/28 11:06:42.314 UTC <Geode Membership View Creator> tid=0x72] finished waiting for responses to view preparation
> [warn 2017/06/28 11:06:42.315 UTC <Geode Membership View Creator> tid=0x72] these members failed to respond to the view change: [asf902(31916)<ec>:8890]
> [info 2017/06/28 11:06:42.315 UTC <Geode View Creator verification thread 1> tid=0x75] checking state of member asf902(31916)<ec>:8890
> [info 2017/06/28 11:06:42.316 UTC <Geode View Creator verification thread 1> tid=0x75] member asf902(31916)<ec>:8890 failed availability check
> [info 2017/06/28 11:06:42.329 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, asf902(31916)<ec>:8888]
> old view is: null
> [info 2017/06/28 11:06:42.329 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:42.330 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v6>:8887
> [info 2017/06/28 11:06:42.330 UTC <Test worker> tid=0x13] received join request from asf902(31916)<ec>:8890
> [info 2017/06/28 11:06:42.330 UTC <Geode Membership View Creator> tid=0x77] View Creator thread is starting
> [info 2017/06/28 11:06:42.331 UTC <Test worker> tid=0x13] received join request from asf902(31916)<ec>:8890
> [info 2017/06/28 11:06:42.331 UTC <Geode Membership View Creator> tid=0x77] preparing new view View[asf902(31916)<ec><v6>:8887|6] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> [info 2017/06/28 11:06:47.332 UTC <Geode Membership View Creator> tid=0x77] finished waiting for responses to view preparation
> [warn 2017/06/28 11:06:47.333 UTC <Geode Membership View Creator> tid=0x77] these members failed to respond to the view change: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:47.334 UTC <Geode View Creator verification thread 1> tid=0x78] checking state of member asf902(31916)<ec>:8889
> [info 2017/06/28 11:06:47.334 UTC <Geode View Creator verification thread 2> tid=0x79] checking state of member asf902(31916)<ec>:8888
> [info 2017/06/28 11:06:47.334 UTC <Geode View Creator verification thread 1> tid=0x78] member asf902(31916)<ec>:8889 passed availability check
> [info 2017/06/28 11:06:47.335 UTC <Geode View Creator verification thread 2> tid=0x79] member asf902(31916)<ec>:8888 passed availability check
> [info 2017/06/28 11:06:47.335 UTC <Geode Membership View Creator> tid=0x77] received new view: View[asf902(31916)<ec><v6>:8887|6] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> old view is: View[asf902(31916)<ec><v6>:8887|1] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> [info 2017/06/28 11:06:47.336 UTC <Geode Membership View Creator> tid=0x77] sending new view View[asf902(31916)<ec><v6>:8887|6] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> [info 2017/06/28 11:06:47.637 UTC <Geode Membership View Creator> tid=0x77] View Creator is processing 2 requests for the next membership view
> [info 2017/06/28 11:06:47.637 UTC <Geode Membership View Creator> tid=0x77] preparing new view View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889, asf902(31916)<ec><v7>:8890]
> [info 2017/06/28 11:06:52.638 UTC <Geode Membership View Creator> tid=0x77] finished waiting for responses to view preparation
> [warn 2017/06/28 11:06:52.639 UTC <Geode Membership View Creator> tid=0x77] these members failed to respond to the view change: [asf902(31916)<ec>:8889, asf902(31916)<ec>:8888, asf902(31916)<ec><v7>:8890]
> [info 2017/06/28 11:06:52.639 UTC <Geode View Creator verification thread 1> tid=0x7a] checking state of member asf902(31916)<ec>:8889
> [info 2017/06/28 11:06:52.640 UTC <Geode View Creator verification thread 3> tid=0x7c] checking state of member asf902(31916)<ec><v7>:8890
> [info 2017/06/28 11:06:52.640 UTC <Geode View Creator verification thread 2> tid=0x7b] checking state of member asf902(31916)<ec>:8888
> [info 2017/06/28 11:06:52.640 UTC <Geode View Creator verification thread 1> tid=0x7a] member asf902(31916)<ec>:8889 passed availability check
> [info 2017/06/28 11:06:52.640 UTC <Geode View Creator verification thread 3> tid=0x7c] member asf902(31916)<ec><v7>:8890 passed availability check
> [info 2017/06/28 11:06:52.641 UTC <Geode View Creator verification thread 2> tid=0x7b] member asf902(31916)<ec>:8888 passed availability check
> [info 2017/06/28 11:06:52.642 UTC <Geode Membership View Creator> tid=0x77] received new view: View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889, asf902(31916)<ec><v7>:8890]
> old view is: View[asf902(31916)<ec><v6>:8887|6] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889]
> [info 2017/06/28 11:06:52.642 UTC <Geode Membership View Creator> tid=0x77] sending new view View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec>:8889, asf902(31916)<ec><v7>:8890]
> [info 2017/06/28 11:06:53.336 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8887|1] members: [asf902(31916)<ec>:8887{lead}, asf902(31916)<ec>:8888]
> old view is: null
> [info 2017/06/28 11:06:53.337 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:53.337 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v6>:8887
> [info 2017/06/28 11:06:53.338 UTC <Geode Membership View Creator> tid=0x7e] View Creator thread is starting
> [info 2017/06/28 11:06:53.339 UTC <Geode Membership View Creator> tid=0x7e] preparing new view View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec><v6>:8889]
> [info 2017/06/28 11:06:55.338 UTC <Geode Membership View Creator> tid=0x7e] finished waiting for responses to view preparation
> [info 2017/06/28 11:06:55.338 UTC <Geode Membership View Creator> tid=0x7e] received new view: View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec><v6>:8889]
> old view is: View[asf902(31916)<ec><v6>:8887|1] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888]
> [info 2017/06/28 11:06:55.340 UTC <Geode Membership View Creator> tid=0x7e] sending new view View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec><v6>:8889]
> new view is View[asf902(31916)<ec><v6>:8887|7] members: [asf902(31916)<ec><v6>:8887{lead}, asf902(31916)<ec>:8888, asf902(31916)<ec><v6>:8889]
> [info 2017/06/28 11:06:55.445 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [info 2017/06/28 11:06:55.445 UTC <Test worker> tid=0x13] received leave request from asf902(31916)<ec>:8888 for asf902(31916)<ec>:8888
> [info 2017/06/28 11:06:55.445 UTC <Test worker> tid=0x13] This member is becoming the membership coordinator with address asf902(31916)<ec>:8887
> [info 2017/06/28 11:06:55.446 UTC <Test worker> tid=0x13] ViewCreator starting on:asf902(31916)<ec><v8>:8887
> [info 2017/06/28 11:06:55.446 UTC <Geode Membership View Creator> tid=0x82] View Creator thread is starting
> [info 2017/06/28 11:06:55.449 UTC <Test worker> tid=0x13] received new view: View[asf902(31916)<ec>:8888|1] members: [asf902(31916)<ec>:8888{lead}, asf902(31916)<ec>:8887]
> old view is: null
> [fatal 2017/06/28 11:06:55.449 UTC <Test worker> tid=0x13] This member is no longer in the membership view.  My ID is asf902(31916)<ec>:8887 and the new view is View[asf902(31916)<ec>:8888|2] members: [asf902(31916)<ec>:8889{lead}, asf902(31916)<ec>:8890, asf902(31916)<ec>:8891]
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message