lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-4.x-Windows (64bit/jdk1.7.0_60) - Build # 4079 - Still Failing!
Date Mon, 07 Jul 2014 08:01:07 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Windows/4079/
Java: 64bit/jdk1.7.0_60 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.OverseerTest.testOverseerFailure

Error Message:
Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed

Stack Trace:
org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
	at __randomizedtesting.SeedInfo.seed([3593263D2DCB1FF7:319BA9CE3F6EF0D6]:0)
	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:143)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
	at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
	at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:661)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
	at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:40)
	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:130)
	... 48 more
Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
	at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:432)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:429)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:386)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:373)
	at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:135)
	at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:34)
	... 49 more




Build Log:
[...truncated 11402 lines...]
   [junit4] Suite: org.apache.solr.cloud.OverseerTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\solr.cloud.OverseerTest-3593263D2DCB1FF7-001\init-core-data-001
   [junit4]   2> 3038481 T9421 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 3038482 T9421 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 3038482 T9421 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 3038487 T9421 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
   [junit4]   2> 3038490 T9421 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3038491 T9422 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 3038640 T9421 oasc.ZkTestServer.run start zk server on port:65465
   [junit4]   2> 3038646 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038658 T9428 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@586552dc name:ZooKeeperConnection Watcher:127.0.0.1:65465 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038658 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038667 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038670 T9430 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@218a1d98 name:ZooKeeperConnection Watcher:127.0.0.1:65465 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038670 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038670 T9421 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 3038679 T9423 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1470fbbb3570001, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 3038680 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038683 T9432 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b809d4d name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038684 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038684 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 3038689 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038692 T9434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@104ff829 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038692 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038693 T9421 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 3038699 T9421 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 3038706 T9421 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:65465_solr
   [junit4]   2> 3038706 T9421 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 3038711 T9421 oasc.Overseer.start Overseer (id=92059616226508803-127.0.0.1:65465_solr-n_0000000000) starting
   [junit4]   2> 3038715 T9421 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 3038733 T9421 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 3038742 T9421 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 3038750 T9421 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 3038759 T9421 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 3038766 T9421 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 3038780 T9436 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 3038781 T9421 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 3038783 T9435 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 3038787 T9421 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 3038793 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3038802 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038806 T9438 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2bab9bb name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038806 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038809 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3038813 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
   [junit4]   2> 3038821 T9432 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 3038822 T9438 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 3038824 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038827 T9440 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8605623 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038828 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038829 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3038834 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 3038840 T9440 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 3038840 T9432 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 3038840 T9438 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 3038843 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038846 T9442 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f7a9b4f name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038847 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038848 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3038853 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
   [junit4]   2> 3038860 T9438 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 3038860 T9432 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 3038860 T9440 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 3038860 T9442 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 3038863 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038866 T9444 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58b9daec name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038866 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038869 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3038873 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
   [junit4]   2> 3038881 T9432 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 3038881 T9438 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 3038881 T9444 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 3038882 T9442 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 3038882 T9440 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 3038884 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038887 T9446 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@156af31c name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038887 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038890 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3038894 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
   [junit4]   2> 3038904 T9446 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 3038904 T9442 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 3038904 T9440 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 3038904 T9444 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 3038906 T9438 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 3038906 T9432 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 3038909 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038913 T9448 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@322d375a name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038913 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038916 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3038923 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
   [junit4]   2> 3038932 T9442 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 3038932 T9448 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 3038932 T9440 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 3038932 T9446 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 3038933 T9432 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 3038933 T9444 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 3038933 T9438 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 3038936 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038939 T9450 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fa47ce0 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038941 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038942 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3038948 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
   [junit4]   2> 3038955 T9444 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 3038957 T9438 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 3038957 T9432 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 3038957 T9442 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 3038957 T9448 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 3038957 T9446 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 3038958 T9440 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 3038958 T9450 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 3038960 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038963 T9452 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d0f9b31 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038964 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038966 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3038973 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
   [junit4]   2> 3038980 T9442 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 3038980 T9448 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 3038982 T9432 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 3038982 T9444 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 3038982 T9438 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 3038982 T9450 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 3038983 T9452 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 3038983 T9446 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 3038983 T9440 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 3038985 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3038988 T9454 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c015330 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3038988 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3038991 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3038995 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
   [junit4]   2> 3039001 T9440 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 3039002 T9450 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 3039002 T9442 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 3039002 T9454 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 3039002 T9438 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 3039004 T9444 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 3039004 T9452 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 3039004 T9448 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 3039004 T9432 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 3039004 T9446 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 3039005 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039008 T9456 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36f9da7b name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039008 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039010 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039014 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
   [junit4]   2> 3039021 T9456 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039023 T9454 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039023 T9450 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039023 T9438 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039023 T9444 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039023 T9452 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039023 T9442 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039024 T9432 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039024 T9448 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039024 T9440 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039024 T9446 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 3039024 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039029 T9458 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@764c8f73 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039029 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039030 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039036 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
   [junit4]   2> 3039042 T9458 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039042 T9432 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039043 T9446 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039045 T9448 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039045 T9454 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039045 T9456 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039046 T9444 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039046 T9438 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039046 T9450 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039046 T9440 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039046 T9452 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039046 T9442 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 3039048 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039051 T9460 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1830e114 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039052 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039054 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039058 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
   [junit4]   2> 3039064 T9446 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039064 T9450 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039067 T9440 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039067 T9458 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039067 T9442 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039067 T9448 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039068 T9460 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039068 T9444 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039068 T9452 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039068 T9454 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039068 T9438 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039068 T9456 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039070 T9432 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 3039070 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039074 T9462 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@373c1a73 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039074 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039077 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039081 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
   [junit4]   2> 3039087 T9440 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039089 T9452 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039089 T9446 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039090 T9444 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039092 T9442 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039092 T9458 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039092 T9448 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039093 T9460 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039093 T9432 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039093 T9438 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039093 T9454 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039093 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039093 T9456 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039093 T9462 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039093 T9450 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 3039098 T9464 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f8c5614 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039098 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039100 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039105 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
   [junit4]   2> 3039115 T9442 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039115 T9448 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039115 T9458 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039115 T9432 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039117 T9456 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039117 T9462 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039117 T9452 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039117 T9444 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039117 T9454 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039117 T9438 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039117 T9464 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039118 T9450 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039118 T9460 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039118 T9446 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039119 T9440 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 3039119 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039122 T9466 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59b56d18 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039124 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039125 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039131 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
   [junit4]   2> 3039141 T9432 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039141 T9458 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039143 T9442 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039143 T9448 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039143 T9464 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039143 T9444 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039143 T9466 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039143 T9452 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039144 T9462 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039144 T9454 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039144 T9438 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039144 T9456 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039144 T9440 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039144 T9446 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039146 T9450 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039146 T9460 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 3039146 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039149 T9468 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5077743 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039150 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039152 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039156 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
   [junit4]   2> 3039163 T9460 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039163 T9444 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039165 T9440 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039165 T9450 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039165 T9446 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039165 T9432 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039166 T9452 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039166 T9468 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039166 T9462 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039166 T9466 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039166 T9438 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039168 T9454 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039168 T9456 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039168 T9464 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039168 T9442 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039168 T9448 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039169 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039171 T9458 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 3039174 T9470 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@787f0857 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039174 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039180 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039185 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
   [junit4]   2> 3039193 T9458 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039193 T9432 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039194 T9444 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039194 T9452 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039197 T9442 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039197 T9464 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039197 T9438 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039199 T9462 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039199 T9448 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039199 T9468 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039199 T9440 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039200 T9454 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039200 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039200 T9446 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039200 T9450 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039200 T9460 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039200 T9466 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039200 T9470 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039200 T9456 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 3039204 T9472 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7edc9d0b name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039206 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039207 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039212 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
   [junit4]   2> 3039221 T9472 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039221 T9442 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039221 T9448 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039222 T9458 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039222 T9432 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039222 T9468 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039222 T9470 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039222 T9456 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039222 T9462 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039223 T9454 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039223 T9444 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039223 T9450 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039223 T9464 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039223 T9440 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039225 T9466 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039225 T9452 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039225 T9460 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039225 T9446 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039225 T9438 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 3039226 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039229 T9474 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d479b24 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039229 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039232 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039237 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
   [junit4]   2> 3039245 T9448 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039245 T9472 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039245 T9442 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039247 T9466 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039248 T9432 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039248 T9462 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039248 T9470 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039248 T9452 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039248 T9474 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039248 T9464 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039250 T9458 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039250 T9468 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039250 T9454 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039250 T9450 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039250 T9444 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039250 T9440 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039250 T9446 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039250 T9438 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039250 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039250 T9460 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039250 T9456 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 3039256 T9476 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fa5775e name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039256 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039257 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039263 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
   [junit4]   2> 3039272 T9458 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039272 T9474 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039273 T9442 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039273 T9440 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039273 T9472 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039273 T9446 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039273 T9460 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039273 T9432 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039275 T9450 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039275 T9448 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039275 T9438 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039275 T9476 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039275 T9464 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039275 T9452 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039276 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039276 T9456 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039276 T9470 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039276 T9468 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039276 T9454 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039276 T9462 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039276 T9466 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039276 T9444 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 3039282 T9478 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f932b6e name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039282 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039285 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039292 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
   [junit4]   2> 3039306 T9460 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039307 T9478 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039310 T9450 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039310 T9454 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039310 T9452 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039311 T9462 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039311 T9472 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039311 T9442 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039311 T9458 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039313 T9476 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039313 T9440 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039313 T9432 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039313 T9466 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039314 T9438 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039314 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039314 T9448 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039314 T9468 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039314 T9474 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039314 T9470 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039314 T9456 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039314 T9446 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039314 T9464 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039314 T9444 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 3039319 T9480 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@176a33d4 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039319 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039323 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039333 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
   [junit4]   2> 3039345 T9480 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039347 T9450 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039347 T9446 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039347 T9440 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039347 T9478 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039347 T9458 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039348 T9460 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039348 T9432 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039348 T9448 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039348 T9472 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039349 T9442 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039351 T9462 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039351 T9452 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039351 T9444 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039352 T9474 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039352 T9470 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039352 T9456 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039352 T9438 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039352 T9468 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039352 T9466 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039352 T9464 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039352 T9476 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039352 T9454 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 3039352 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039357 T9482 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79426208 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039358 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039360 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039364 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
   [junit4]   2> 3039376 T9474 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039377 T9470 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039379 T9456 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039379 T9468 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039380 T9432 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039382 T9480 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039382 T9458 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039383 T9442 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039383 T9476 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039383 T9482 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039385 T9448 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039385 T9438 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039385 T9472 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039385 T9454 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039386 T9462 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039386 T9452 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039386 T9466 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039386 T9444 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039388 T9464 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039388 T9450 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039388 T9440 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039388 T9446 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039388 T9478 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039388 T9460 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 3039388 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039395 T9484 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ce058ab name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039395 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039398 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039404 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
   [junit4]   2> 3039415 T9468 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039417 T9470 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039418 T9474 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039420 T9444 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039421 T9464 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039421 T9466 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039421 T9454 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039421 T9472 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039421 T9476 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039423 T9458 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039423 T9462 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039423 T9448 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039423 T9452 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039423 T9478 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039424 T9446 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039424 T9460 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039424 T9484 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039424 T9482 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039424 T9480 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039424 T9442 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039426 T9438 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039426 T9432 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039426 T9456 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039426 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039426 T9450 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039426 T9440 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 3039431 T9486 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e9f2875 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039431 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039434 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039440 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node24
   [junit4]   2> 3039453 T9486 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039453 T9478 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039453 T9460 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039455 T9440 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039455 T9446 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039455 T9454 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039455 T9450 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039456 T9444 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039456 T9464 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039458 T9466 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039459 T9470 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039459 T9442 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039459 T9480 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039459 T9472 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039461 T9438 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039461 T9462 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039461 T9448 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039462 T9482 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039462 T9484 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039464 T9474 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039465 T9458 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039465 T9468 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039465 T9432 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039465 T9452 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039467 T9456 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039467 T9476 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 3039468 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039472 T9488 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@259778f7 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039474 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039477 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039489 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node25
   [junit4]   2> 3039503 T9442 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039505 T9448 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039505 T9432 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039505 T9456 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039506 T9484 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039508 T9488 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039509 T9462 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039509 T9472 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039509 T9458 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039509 T9464 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039511 T9454 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039511 T9474 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039511 T9468 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039511 T9482 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039511 T9480 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039511 T9438 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039512 T9452 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039512 T9476 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039512 T9470 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039512 T9444 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039512 T9466 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039512 T9440 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039514 T9446 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039514 T9450 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039514 T9478 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039514 T9460 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039514 T9486 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 3039516 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039519 T9490 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c6719cf name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039521 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039524 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039530 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node26
   [junit4]   2> 3039540 T9452 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039540 T9462 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039541 T9438 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039541 T9456 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039541 T9484 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039541 T9432 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039541 T9448 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039541 T9442 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039544 T9480 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039544 T9488 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039546 T9482 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039546 T9478 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039546 T9490 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039546 T9450 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039546 T9440 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039546 T9446 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039547 T9474 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039547 T9466 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039547 T9454 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039547 T9470 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039549 T9472 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039549 T9468 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039550 T9476 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039550 T9444 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039552 T9464 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039552 T9460 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039552 T9486 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039552 T9458 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 3039555 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039560 T9492 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b09a8d0 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039560 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039566 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039574 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node27
   [junit4]   2> 3039585 T9492 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039585 T9488 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039585 T9468 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039587 T9454 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039587 T9490 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039587 T9464 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039587 T9458 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039587 T9446 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039588 T9480 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039588 T9460 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039588 T9442 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039588 T9452 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039588 T9438 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039590 T9472 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039590 T9456 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039590 T9470 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039590 T9478 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039590 T9432 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039590 T9450 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039591 T9474 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039591 T9476 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039591 T9444 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039591 T9486 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039591 T9466 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039591 T9484 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039593 T9440 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039593 T9462 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039593 T9482 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039593 T9448 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 3039594 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039600 T9494 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26363404 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039601 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039604 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039610 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node28
   [junit4]   2> 3039625 T9432 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039626 T9448 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039626 T9442 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039626 T9452 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039628 T9462 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039628 T9438 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039628 T9484 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039628 T9456 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039629 T9492 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039629 T9480 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039631 T9488 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039631 T9482 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039632 T9486 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039632 T9490 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039632 T9458 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039632 T9468 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039632 T9470 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039632 T9454 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039634 T9446 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039634 T9464 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039634 T9476 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039634 T9466 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039634 T9494 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039635 T9472 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039635 T9440 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039635 T9478 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039635 T9474 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039637 T9450 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039637 T9444 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039637 T9460 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 3039639 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039644 T9496 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2a10c128 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039645 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039648 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039659 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node29
   [junit4]   2> 3039680 T9488 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039680 T9482 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039680 T9480 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039686 T9496 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039688 T9456 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039688 T9460 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039688 T9438 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039688 T9450 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039691 T9432 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039691 T9494 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039691 T9452 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039691 T9442 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039691 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039692 T9448 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9474 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9458 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9492 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9468 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9476 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9462 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9470 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9472 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9454 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9466 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9490 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9464 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9444 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9446 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9484 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9486 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9440 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039692 T9478 oascc.ZkStateReader$3.process Updating live nodes... (30)
   [junit4]   2> 3039698 T9498 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e3446b8 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039702 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039705 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039711 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node30
   [junit4]   2> 3039723 T9458 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039723 T9490 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039724 T9476 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039724 T9472 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039724 T9454 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039724 T9466 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039727 T9444 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039727 T9470 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039727 T9486 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039729 T9440 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039729 T9464 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039729 T9474 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039730 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039730 T9468 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039730 T9460 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039730 T9446 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039732 T9456 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039730 T9478 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039732 T9480 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039733 T9462 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039730 T9450 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039730 T9494 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039733 T9448 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039735 T9496 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039736 T9452 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039736 T9492 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039736 T9482 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039738 T9442 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039738 T9484 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039738 T9488 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039738 T9438 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039739 T9432 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039739 T9498 oascc.ZkStateReader$3.process Updating live nodes... (31)
   [junit4]   2> 3039742 T9500 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cb23b76 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039743 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039746 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039751 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node31
   [junit4]   2> 3039758 T9490 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039758 T9458 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039764 T9476 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039764 T9446 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039764 T9470 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039764 T9462 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039765 T9468 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039765 T9442 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039765 T9460 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039765 T9450 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039765 T9444 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039765 T9464 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039767 T9432 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039767 T9440 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039767 T9456 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039767 T9472 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039767 T9438 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039767 T9486 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9492 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9488 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9480 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9500 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9466 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9496 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9474 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9448 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9498 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9494 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9454 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9478 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9484 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9482 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039768 T9452 oascc.ZkStateReader$3.process Updating live nodes... (32)
   [junit4]   2> 3039767 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039773 T9502 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32d1f5f7 name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039774 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039777 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039786 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node32
   [junit4]   2> 3039795 T9452 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039801 T9500 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039801 T9502 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039801 T9438 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039801 T9456 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039801 T9488 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039802 T9442 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039802 T9484 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039802 T9482 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039802 T9432 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039802 T9492 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039802 T9496 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039804 T9480 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039804 T9462 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039804 T9448 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039804 T9498 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039804 T9444 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039804 T9464 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039805 T9472 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039805 T9466 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039806 T9450 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039806 T9478 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039806 T9474 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039806 T9476 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039806 T9454 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039806 T9490 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039808 T9458 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039808 T9486 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039808 T9440 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039808 T9446 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039809 T9421 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3039809 T9494 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039811 T9460 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039811 T9468 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039811 T9470 oascc.ZkStateReader$3.process Updating live nodes... (33)
   [junit4]   2> 3039817 T9504 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@546c0b7c name:ZooKeeperConnection Watcher:127.0.0.1:65465/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3039817 T9421 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3039820 T9421 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3039826 T9421 oascc.SolrZkClient.makePath makePath: /live_nodes/node33
   [junit4]   2> 3039836 T9472 oascc.ZkStateReader$3.process Updating live nodes... (34)
   [junit4]   2> 3039837 T9476 oascc.ZkStateReader$3.process Updating live nodes... (34)
   [junit4]   2> 3039837 T9458 oascc.ZkStateReader$3.process Updating live nodes... (34)
   [junit4]   2> 3039837 T9490 oascc.ZkStateReader$3.process Updating live nodes... (34)
   [junit4]   2> 3039837 T9470 oascc.ZkStateReader$3.process Updating live nodes... (34)
   [junit4]   2> 3039839 T9468 oascc.ZkStateReader$3.process Updating live nodes... (34)
   [junit4]   2> 3039840 T9474 oascc.ZkStateReader$3.process Updating live nodes... (34)
   [junit4]   2> 3039840 T9466 oascc.ZkStateReader$3.process Updating live nodes... (34)
   [junit4]   2> 3039840 T9454 oascc.ZkStateReader$3.process Updating live nodes... (34)
   [junit4]   2> 3039842 T9446 oascc.ZkStateReade

[...truncated too long message...]

unit4]   2> 3105473 T9791 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:49450 49450
   [junit4]   2> 3104862 T9806 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer main queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue-work
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.orderedChildren(DistributedQueue.java:99)
   [junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.remove(DistributedQueue.java:204)
   [junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.poll(DistributedQueue.java:625)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:264)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 3105475 T9806 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:49450_solr
   [junit4]   2> 3105475 T9421 oas.SolrTestCaseJ4.tearDown ###Ending testDoubleAssignment
   [junit4]   2> 3105478 T9810 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
   [junit4]   2> 
   [junit4]   2> 3105479 T9421 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 3105479 T9421 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 3107871 T9421 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\solr-core\test\J0\.\solr.cloud.OverseerTest-3593263D2DCB1FF7-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=es_SV, timezone=Etc/GMT-12
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=2,threads=1,free=74088480,total=449646592
   [junit4]   2> NOTE: All tests run in this JVM: [TestComplexPhraseQParserPlugin, BlockDirectoryTest, OpenExchangeRatesOrgProviderTest, TestDocSet, DistributedQueryComponentOptimizationTest, TestFieldTypeCollectionResource, TestRandomMergePolicy, DirectSolrConnectionTest, TestCollectionAPI, DistribCursorPagingTest, TestInitQParser, TestCollationFieldDocValues, FastVectorHighlighterTest, TestCopyFieldCollectionResource, DirectUpdateHandlerOptimizeTest, PingRequestHandlerTest, XsltUpdateRequestHandlerTest, ZkCLITest, UnloadDistributedZkTest, TestManagedResourceStorage, PrimUtilsTest, CoreMergeIndexesAdminHandlerTest, SolrXmlInZkTest, TestLeaderElectionZkExpiry, TestSweetSpotSimilarityFactory, TestSolrQueryParserResource, ShardSplitTest, TestSolrXmlPersistor, SolrCoreTest, TestSchemaResource, TestPerFieldSimilarity, TestDynamicFieldResource, RegexBoostProcessorTest, TestRemoteStreaming, HighlighterTest, SSLMigrationTest, TestRecovery, TestReload, TestCloudInspectUtil, RemoteQueryErrorTest, TestIndexingPerformance, HdfsLockFactoryTest, ZkNodePropsTest, SpatialFilterTest, MoreLikeThisHandlerTest, TestBinaryField, UpdateRequestProcessorFactoryTest, TestPostingsSolrHighlighter, TestStressLucene, TestBadConfig, TestDynamicFieldCollectionResource, MBeansHandlerTest, TestOverriddenPrefixQueryForCustomFieldType, TestRangeQuery, DisMaxRequestHandlerTest, TestCloudManagedSchemaConcurrent, CircularListTest, SimplePostToolTest, HdfsSyncSliceTest, TestSolr4Spatial, TestFileDictionaryLookup, TestReRankQParserPlugin, ZkSolrClientTest, FieldMutatingUpdateProcessorTest, TestSolrCoreProperties, PrimitiveFieldTypeTest, PolyFieldTest, CopyFieldTest, ExternalFileFieldSortTest, SolrInfoMBeanTest, TestCodecSupport, TestHighFrequencyDictionaryFactory, AddBlockUpdateTest, DistribDocExpirationUpdateProcessorTest, TestSchemaNameResource, TestSurroundQueryParser, SimpleFacetsTest, FieldAnalysisRequestHandlerTest, HighlighterConfigTest, ChaosMonkeySafeLeaderTest, DistributedTermsComponentTest, TestAnalyzeInfixSuggestions, XmlUpdateRequestHandlerTest, TestCollationKeyRangeQueries, DocValuesMissingTest, TestPhraseSuggestions, TriLevelCompositeIdRoutingTest, SyncSliceTest, SpellPossibilityIteratorTest, SolrIndexConfigTest, TestSolrDeletionPolicy2, TestRequestStatusCollectionAPI, SolrCoreCheckLockOnStartupTest, AddSchemaFieldsUpdateProcessorFactoryTest, UniqFieldsUpdateProcessorFactoryTest, AutoCommitTest, TestSystemIdResolver, TestLFUCache, TestXIncludeConfig, TestConfigSets, TestOmitPositions, ResponseHeaderTest, PeerSyncTest, TestNonDefinedSimilarityFactory, TestStressVersions, BasicZkTest, TestCoreContainer, TestFunctionQuery, TestQueryUtils, ParsingFieldUpdateProcessorsTest, TestDistributedGrouping, HdfsRecoveryZkTest, TestDistributedSearch, TestCSVLoader, CursorMarkTest, CoreAdminCreateDiscoverTest, FullSolrCloudDistribCmdsTest, SolrRequestParserTest, TestManagedSynonymFilterFactory, ChaosMonkeyNothingIsSafeTest, TestFreeTextSuggestions, TestQuerySenderListener, TestHighlightDedupGrouping, MultiThreadedOCPTest, SuggesterTest, MultiTermTest, TestElisionMultitermQuery, HdfsDirectoryTest, ReplicationFactorTest, AnalysisAfterCoreReloadTest, ResourceLoaderTest, TestReversedWildcardFilterFactory, HardAutoCommitTest, NotRequiredUniqueKeyTest, AtomicUpdatesTest, SearchHandlerTest, TermsComponentTest, ClusterStateTest, TestDistributedMissingSort, TestRecoveryHdfs, TestSerializedLuceneMatchVersion, TestCharFilters, PathHierarchyTokenizerFactoryTest, LoggingHandlerTest, TestUtils, TestShardHandlerFactory, OverseerStatusTest, TestDefaultSimilarityFactory, TestSearcherReuse, TestQuerySenderNoQuery, CachingDirectoryFactoryTest, TermVectorComponentDistributedTest, TestTolerantSearch, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, TestJmxIntegration, TestIndexSearcher, BadComponentTest, TestComponentsName, SolrCmdDistributorTest, PreAnalyzedFieldTest, TestLMJelinekMercerSimilarityFactory, DocExpirationUpdateProcessorFactoryTest, SolrIndexSplitterTest, DistributedSpellCheckComponentTest, DistributedQueryComponentCustomSortTest, TestNoOpRegenerator, TestLazyCores, RequestHandlersTest, TestManagedSchema, SystemInfoHandlerTest, TestFieldCollectionResource, TestStandardQParsers, BasicFunctionalityTest, TestExpandComponent, DOMUtilTest, TestManagedStopFilterFactory, TestSolrQueryParser, SuggesterWFSTTest, FileBasedSpellCheckerTest, TestReplicationHandler, TestBM25SimilarityFactory, ClusterStateUpdateTest, DistributedSuggestComponentTest, RecoveryZkTest, TestFastLRUCache, TestMultiCoreConfBootstrap, TestTrie, TestRandomFaceting, HighlighterMaxOffsetTest, TestPseudoReturnFields, TestCollationField, TestCoreDiscovery, TestSolrIndexConfig, TestFieldResource, TestWriterPerf, LukeRequestHandlerTest, TestManagedResource, BasicDistributedZkTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, OverseerTest]
   [junit4] Completed in 69.43s, 9 tests, 1 error <<< FAILURES!

[...truncated 619 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:467: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:447: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:45: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\extra-targets.xml:37: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\common-build.xml:496: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\common-build.xml:1299: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\common-build.xml:923: There were test failures: 405 suites, 1656 tests, 1 error, 42 ignored (21 assumptions)

Total time: 116 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.7.0_60 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Mime
View raw message