lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_51) - Build # 3934 - Failure!
Date Tue, 08 Apr 2014 08:58:51 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/3934/
Java: 64bit/jdk1.7.0_51 -XX:-UseCompressedOops -XX:+UseParallelGC -XX:-UseSuperWord

1 tests failed.
REGRESSION:  org.apache.solr.cloud.OverseerTest.testOverseerFailure

Error Message:
Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed

Stack Trace:
org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
	at __randomizedtesting.SeedInfo.seed([2CADCFC56BA6D7F5:28A54036790338D4]:0)
	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:136)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:134)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
	at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
	at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:661)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
	at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:432)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:429)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:386)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:373)
	at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:128)
	at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:123)
	... 46 more




Build Log:
[...truncated 11066 lines...]
   [junit4] Suite: org.apache.solr.cloud.OverseerTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solr.cloud.OverseerTest-2CADCFC56BA6D7F5-001\init-core-data-001
   [junit4]   2> 2632051 T8631 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 2632051 T8631 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 2632051 T8631 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 2632056 T8631 oas.SolrTestCaseJ4.setUp ###Starting testPlaceholders
   [junit4]   2> 2632059 T8631 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 2632060 T8632 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2632174 T8631 oasc.ZkTestServer.run start zk server on port:54315
   [junit4]   2> 2632178 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2632188 T8638 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@690eefb5 name:ZooKeeperConnection Watcher:127.0.0.1:54315/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2632188 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2632191 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2632195 T8640 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a636ef6 name:ZooKeeperConnection Watcher:127.0.0.1:54315 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2632195 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2632200 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2632203 T8642 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@592c22fc name:ZooKeeperConnection Watcher:127.0.0.1:54315 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2632203 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2632203 T8631 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2632210 T8631 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2632214 T8631 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2632218 T8631 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2632221 T8631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2632228 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2632231 T8644 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3049adfc name:ZooKeeperConnection Watcher:127.0.0.1:54315/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2632231 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2632233 T8631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2632238 T8631 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 2632244 T8638 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 2632244 T8644 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 2632247 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2632250 T8646 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b32fe6c name:ZooKeeperConnection Watcher:127.0.0.1:54315/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2632250 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2632251 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2632255 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2632263 T8631 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:54315_solr
   [junit4]   2> 2632263 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2632268 T8631 oasc.Overseer.start Overseer (id=91550253298614276-127.0.0.1:54315_solr-n_0000000000) starting
   [junit4]   2> 2632271 T8631 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2632277 T8631 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2632283 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2632289 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2632295 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2632302 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2632313 T8648 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2632316 T8648 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:54315_solr
   [junit4]   2> 2632317 T8647 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2632321 T8647 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2632322 T8647 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=12 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"12",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 2632322 T8647 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6, shard7, shard8, shard9, shard10, shard11, shard12]
   [junit4]   2> 2632322 T8647 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 2632328 T8646 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2632330 T8638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2632330 T8644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2632945 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 2632985 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 2633011 T8646 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2633013 T8631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2633016 T8647 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2633024 T8644 oascc.ZkStateReader$3.process Updating live nodes... (0)
   [junit4]   2> 2633025 T8638 oascc.ZkStateReader$3.process Updating live nodes... (0)
   [junit4]   2> 2633030 T8633 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145407866f60003, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2633036 T8633 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145407866f60000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2633039 T8631 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54315 54315
   [junit4]   2> 2634900 T8647 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer main queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue-work/qn-
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:243)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:240)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:240)
   [junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:311)
   [junit4]   2> 	at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:295)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:254)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2635862 T8647 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:54315_solr
   [junit4]   2> 2635869 T8632 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54315 54315
   [junit4]   2> 2635871 T8649 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:302)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:291)
   [junit4]   2> 
   [junit4]   2> 2635878 T8631 oas.SolrTestCaseJ4.tearDown ###Ending testPlaceholders
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2635900 T8631 oas.SolrTestCaseJ4.setUp ###Starting testReplay
   [junit4]   2> 2635902 T8631 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 2635904 T8650 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2636031 T8631 oasc.ZkTestServer.run start zk server on port:54333
   [junit4]   2> 2636036 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2636049 T8656 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78cf220a name:ZooKeeperConnection Watcher:127.0.0.1:54333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2636050 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2636054 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2636060 T8658 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7828c41a name:ZooKeeperConnection Watcher:127.0.0.1:54333 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2636060 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2636074 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2636080 T8660 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20f9c652 name:ZooKeeperConnection Watcher:127.0.0.1:54333 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2636080 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2636080 T8631 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2636097 T8631 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2636107 T8631 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2636115 T8631 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2636125 T8631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2636138 T8631 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2636159 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2636164 T8662 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6382e2f8 name:ZooKeeperConnection Watcher:127.0.0.1:54333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2636165 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2636168 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2636179 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2636196 T8631 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:54333_solr
   [junit4]   2> 2636197 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2636207 T8631 oasc.Overseer.start Overseer (id=91550253550927875-127.0.0.1:54333_solr-n_0000000000) starting
   [junit4]   2> 2636217 T8631 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2636233 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2636244 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2636251 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2636260 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2636273 T8664 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2636274 T8664 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:54333_solr
   [junit4]   2> 2636277 T8663 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2636278 T8663 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
   [junit4]   2> 2636279 T8663 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "shard":"s1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "roles":"",
   [junit4]   2> 	  "state":"recovering"}
   [junit4]   2> 2636282 T8656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 2636288 T8663 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "shard":"s1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core":"core2",
   [junit4]   2> 	  "roles":"",
   [junit4]   2> 	  "state":"recovering"}
   [junit4]   2> 2636290 T8656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 2636294 T8663 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2636296 T8663 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2636298 T8663 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "shard":"s1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core":"core3",
   [junit4]   2> 	  "roles":"",
   [junit4]   2> 	  "state":"recovering"}
   [junit4]   2> 2636302 T8662 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2636305 T8656 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 2636408 T8651 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145407876000003, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2636419 T8651 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145407876000000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2636419 T8631 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54333 54333
   [junit4]   2> 2639244 T8650 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54333 54333
   [junit4]   2> 2639248 T8631 oas.SolrTestCaseJ4.tearDown ###Ending testReplay
   [junit4]   2> 2639249 T8663 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:54333_solr
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2639252 T8665 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:302)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:291)
   [junit4]   2> 
   [junit4]   2> 2639260 T8631 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
   [junit4]   2> 2639263 T8631 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 2639264 T8666 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2639390 T8631 oasc.ZkTestServer.run start zk server on port:54348
   [junit4]   2> 2639395 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2639407 T8672 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f17c3a7 name:ZooKeeperConnection Watcher:127.0.0.1:54348/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2639408 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2639412 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2639417 T8674 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2906e74f name:ZooKeeperConnection Watcher:127.0.0.1:54348 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2639417 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2639423 T8667 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1454078831c0001, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2639427 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2639434 T8676 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3721a057 name:ZooKeeperConnection Watcher:127.0.0.1:54348 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2639434 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2639435 T8631 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2639444 T8631 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2639445 T8667 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1454078831c0002, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2639451 T8631 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2639458 T8631 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2639465 T8631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2639471 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2639476 T8678 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ba526b8 name:ZooKeeperConnection Watcher:127.0.0.1:54348/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2639476 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2639479 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2639483 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2639491 T8631 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:54348_solr
   [junit4]   2> 2639491 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2639496 T8631 oasc.Overseer.start Overseer (id=91550253770866691-127.0.0.1:54348_solr-n_0000000000) starting
   [junit4]   2> 2639500 T8631 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2639506 T8631 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2639512 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2639518 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2639524 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2639537 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2639551 T8680 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2639553 T8680 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:54348_solr
   [junit4]   2> 2639556 T8679 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2639564 T8631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2639569 T8679 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2639570 T8679 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "roles":"",
   [junit4]   2> 	  "state":"recovering"}
   [junit4]   2> 2639570 T8679 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2639575 T8678 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2639577 T8672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 2639633 T8631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2639641 T8678 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2639648 T8679 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2639651 T8679 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "roles":"",
   [junit4]   2> 	  "state":"active"}
   [junit4]   2> 2639651 T8679 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
   [junit4]   2> 2639651 T8679 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 2639664 T8678 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2639801 T8672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
   [junit4]   2> 2639838 T8667 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1454078831c0000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2639845 T8631 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54348 54348
   [junit4]   2> 2642668 T8666 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54348 54348
   [junit4]   2> 2642672 T8631 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
   [junit4]   2> 2642672 T8679 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:54348_solr
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2642673 T8681 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:302)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:291)
   [junit4]   2> 
   [junit4]   2> 2642682 T8631 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignment
   [junit4]   2> 2642684 T8631 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 2642684 T8682 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2642811 T8631 oasc.ZkTestServer.run start zk server on port:54363
   [junit4]   2> 2642814 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2642826 T8688 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63a379c6 name:ZooKeeperConnection Watcher:127.0.0.1:54363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2642826 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2642833 T8683 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145407890780000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2642837 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2642843 T8690 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c98090f name:ZooKeeperConnection Watcher:127.0.0.1:54363 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2642843 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2642846 T8631 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2642858 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2642862 T8692 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@694f8f3 name:ZooKeeperConnection Watcher:127.0.0.1:54363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2642862 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2642863 T8631 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2642871 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2642875 T8694 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c666a66 name:ZooKeeperConnection Watcher:127.0.0.1:54363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2642876 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2642877 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2642881 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2642890 T8631 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:54363_solr
   [junit4]   2> 2642890 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2642893 T8631 oasc.Overseer.start Overseer (id=91550253994999811-127.0.0.1:54363_solr-n_0000000000) starting
   [junit4]   2> 2642898 T8631 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2642906 T8631 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2642912 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2642918 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2642925 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2642935 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2642950 T8631 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2642950 T8695 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2642951 T8696 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2642952 T8696 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:54363_solr
   [junit4]   2> 2642953 T8631 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2642957 T8631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2642963 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2642966 T8698 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ce78d86 name:ZooKeeperConnection Watcher:127.0.0.1:54363/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2642967 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2642968 T8631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2642973 T8631 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
   [junit4]   2> 2642978 T8698 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 2642978 T8692 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 2642983 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2642985 T8695 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2642986 T8695 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "node_name":"127.0.0.1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 2642986 T8695 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
   [junit4]   2> 2642986 T8695 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 2642991 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2642992 T8692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2642994 T8698 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2643609 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 2643630 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 2643642 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2643647 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2643647 T8695 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2643656 T8695 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "node_name":"127.0.0.1",
   [junit4]   2> 	  "core":"core2",
   [junit4]   2> 	  "core_node_name":"node2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 2643656 T8695 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 2643656 T8695 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
   [junit4]   2> 2643663 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2643795 T8692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2643795 T8698 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2644274 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
   [junit4]   2> 2644305 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
   [junit4]   2> 2644330 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2644330 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2644330 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2644339 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2644340 T8695 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2644358 T8695 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "node_name":"127.0.0.1",
   [junit4]   2> 	  "core":"core3",
   [junit4]   2> 	  "core_node_name":"node3",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 2644358 T8695 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 2644358 T8695 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2644369 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2644505 T8692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2644506 T8698 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2644937 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 2644965 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2644986 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2644986 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2644987 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2644994 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2644995 T8695 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2645011 T8695 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "node_name":"127.0.0.1",
   [junit4]   2> 	  "core":"core4",
   [junit4]   2> 	  "core_node_name":"node4",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 2645011 T8695 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 2645011 T8695 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 2645022 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2645129 T8698 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2645130 T8692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2645495 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 2645516 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2645516 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2645516 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2645521 T8695 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2645522 T8695 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "node_name":"127.0.0.1",
   [junit4]   2> 	  "core":"core5",
   [junit4]   2> 	  "core_node_name":"node5",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 2645522 T8695 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 2645522 T8695 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
   [junit4]   2> 2645528 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2645634 T8698 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2645634 T8692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2646016 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
   [junit4]   2> 2646035 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2646035 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2646036 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2646038 T8695 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2646039 T8695 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "node_name":"127.0.0.1",
   [junit4]   2> 	  "core":"core6",
   [junit4]   2> 	  "core_node_name":"node6",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"3",
   [junit4]   2> 	  "base_url":"http://127.0.0.1/solr/"}
   [junit4]   2> 2646039 T8695 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
   [junit4]   2> 2646039 T8695 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2646044 T8694 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2646151 T8698 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2646151 T8692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2646535 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 2646562 T8683 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145407890780002, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2646563 T8685 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception:  java.nio.channels.CancelledKeyException
   [junit4]   2> 	at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
   [junit4]   2> 	at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1118)
   [junit4]   2> 	at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
   [junit4]   2> 	at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
   [junit4]   2> 	at org.apache.zookeeper.server.DataTree.deleteNode(DataTree.java:594)
   [junit4]   2> 	at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:802)
   [junit4]   2> 	at org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
   [junit4]   2> 	at org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:994)
   [junit4]   2> 	at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
   [junit4]   2> 	at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
   [junit4]   2> 	at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
   [junit4]   2> 
   [junit4]   2> 2646566 T8698 oascc.ZkStateReader$3.process Updating live nodes... (0)
   [junit4]   2> 2646575 T8631 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54363 54363
   [junit4]   2> 2648068 T8698 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> 2648827 T8698 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard3/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> 2648828 T8682 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54363 54363
   [junit4]   2> 2648828 T8698 oasc.LeaderElector$ElectionWatcher.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> 2648834 T8631 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignment
   [junit4]   2> 2648836 T8695 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:54363_solr
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2648843 T8699 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:302)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:291)
   [junit4]   2> 
   [junit4]   2> 2648850 T8631 oas.SolrTestCaseJ4.setUp ###Starting testOverseerFailure
   [junit4]   2> 2648852 T8631 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 2648853 T8700 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2648953 T8631 oasc.ZkTestServer.run start zk server on port:54381
   [junit4]   2> 2648957 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2648970 T8706 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72949a6 name:ZooKeeperConnection Watcher:127.0.0.1:54381 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2648971 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2648977 T8701 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1454078a8900000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2648980 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2648985 T8708 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61415c63 name:ZooKeeperConnection Watcher:127.0.0.1:54381 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2648986 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2648986 T8631 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2648999 T8701 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1454078a8900001, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2649001 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2649006 T8710 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14d2f9ab name:ZooKeeperConnection Watcher:127.0.0.1:54381/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2649007 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2649007 T8631 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2649016 T8631 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2649024 T8631 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2649030 T8631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2649041 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2649045 T8712 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c35f8b5 name:ZooKeeperConnection Watcher:127.0.0.1:54381/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2649046 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2649049 T8631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2649058 T8631 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 2649069 T8712 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 2649070 T8710 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 2649074 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2649079 T8714 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ead1dd7 name:ZooKeeperConnection Watcher:127.0.0.1:54381/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2649079 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2649081 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2649090 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2649103 T8631 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:54381_solr
   [junit4]   2> 2649104 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2649113 T8631 oasc.Overseer.start Overseer (id=91550254399225860-127.0.0.1:54381_solr-n_0000000000) starting
   [junit4]   2> 2649120 T8631 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2649132 T8631 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2649142 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2649153 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2649161 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2649174 T8631 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2649194 T8716 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2649196 T8716 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:54381_solr
   [junit4]   2> 2649197 T8715 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2650198 T8714 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2650201 T8715 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2650202 T8715 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 2650202 T8715 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
   [junit4]   2> 2650202 T8715 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2650206 T8714 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2650208 T8710 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2650208 T8712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2650699 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 2650730 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2650752 T8714 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2650754 T8631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2650756 T8715 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2650766 T8714 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2650767 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 2650771 T8715 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 2650772 T8715 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 2650779 T8714 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2650881 T8712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2650882 T8710 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2650938 T8701 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1454078a8900004, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2651947 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 2651971 T8631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2651975 T8718 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e6b66a7 name:ZooKeeperConnection Watcher:127.0.0.1:54381/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2651976 T8631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2651977 T8715 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:54381_solr
   [junit4]   2> 2651986 T8719 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:302)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:291)
   [junit4]   2> 
   [junit4]   2> 2651995 T8631 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:54381_solr
   [junit4]   2> 2651995 T8631 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2652004 T8631 oasc.Overseer.start Overseer (id=91550254399225861-127.0.0.1:54381_solr-n_0000000001) starting
   [junit4]   2> 2652046 T8721 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2652049 T8721 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:54381_solr
   [junit4]   2> 2652049 T8720 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2652053 T8720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2652054 T8720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 2652054 T8720 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 2652057 T8718 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2652059 T8712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2652059 T8710 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2652118 T8718 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2652118 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 2652120 T8720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2652129 T8712 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2652130 T8631 oasc.LeaderElector.checkIfIamLeader WARN Failed setting watch org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election/91550254399225859-node1_core1-n_0000000002
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:127)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:661)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 2652132 T8720 oasc.SolrException.log ERROR Problem cleaning up collection in zk:collection1:org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:125)
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 		at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:177)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:174)
   [junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:174)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:631)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:626)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:626)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:626)
   [junit4]   2> 		at org.apache.solr.cloud.Overseer$ClusterStateUpdater.removeCore(Overseer.java:1069)
   [junit4]   2> 		at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:342)
   [junit4]   2> 		at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:241)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 	
   [junit4]   2> 2652134 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2652137 T8718 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2652140 T8718 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2652142 T8712 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2652142 T8720 oasc.Overseer$ClusterStateUpdater.setShardLeader ERROR Could not mark shard leader for non existing collection:collection1
   [junit4]   2> 2652147 T8718 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2652151 T8718 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2652152 T8720 oasc.Overseer$ClusterStateUpdater.setShardLeader ERROR Could not mark shard leader for non existing collection:collection1
   [junit4]   2> 2652156 T8718 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2652258 T8710 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2652258 T8712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2653141 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2654152 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2655160 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2656165 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2657176 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2658187 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2659192 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2660198 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2661202 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2662213 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2663224 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2664234 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2665246 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2666253 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2667259 T8631 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2667274 T8712 oascc.ZkStateReader$3.process Updating live nodes... (0)
   [junit4]   2> 2667275 T8710 oascc.ZkStateReader$3.process Updating live nodes... (0)
   [junit4]   2> 2667277 T8701 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1454078a8900003, likely client ha

[...truncated too long message...]

N  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> 2694733 T8950 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 2694732 T8822 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54501 54501
   [junit4]   2> 2694762 T8631 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignmentBigger
   [junit4]   2> 2694765 T8835 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:54501_solr
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2694766 T8631 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 2694766 T8631 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 2694772 T9011 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:302)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:291)
   [junit4]   2> 
   [junit4]   2> 2697841 T8631 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\.\solr.cloud.OverseerTest-2CADCFC56BA6D7F5-001
   [junit4]   2> NOTE: test params are: codec=Lucene42, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar, timezone=Atlantic/Faeroe
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_51 (64-bit)/cpus=2,threads=1,free=25997864,total=526385152
   [junit4]   2> NOTE: All tests run in this JVM: [ConvertedLegacyTest, ParsingFieldUpdateProcessorsTest, SolrIndexConfigTest, SliceStateUpdateTest, DefaultValueUpdateProcessorTest, TestMaxScoreQueryParser, DistributedQueryComponentCustomSortTest, SpellCheckComponentTest, DOMUtilTest, CurrencyFieldOpenExchangeTest, TestPHPSerializedResponseWriter, ChangedSchemaMergeTest, BadCopyFieldTest, SearchHandlerTest, TestQuerySenderNoQuery, HdfsChaosMonkeySafeLeaderTest, QueryResultKeyTest, TestExpandComponent, TestRangeQuery, TestReload, QueryParsingTest, StressHdfsTest, AlternateDirectoryTest, ZkCLITest, MultiTermTest, TestSchemaSimilarityResource, EchoParamsTest, TestElisionMultitermQuery, TestInitQParser, CachingDirectoryFactoryTest, TestJmxMonitoredMap, TestSolrDeletionPolicy2, TestDocumentBuilder, TestRecoveryHdfs, QueryFacetTest, EnumFieldTest, CursorMarkTest, FunctionTest, TestSolrQueryParserDefaultOperatorResource, ResourceLoaderTest, TestAnalyzeInfixSuggestions, SystemInfoHandlerTest, SampleTest, TestMiniSolrCloudCluster, InfoHandlerTest, TestStressRecovery, SignatureUpdateProcessorFactoryTest, TestPartialUpdateDeduplication, TimeZoneUtilsTest, TestRealTimeGet, TestFastWriter, OpenExchangeRatesOrgProviderTest, DocValuesMultiTest, JSONWriterTest, RegexBoostProcessorTest, AssignTest, TestFreeTextSuggestions, CollectionsAPIAsyncDistributedZkTest, TestFieldTypeResource, TestAnalyzedSuggestions, DistributedSuggestComponentTest, DateMathParserTest, ShardRoutingCustomTest, TestCollapseQParserPlugin, XsltUpdateRequestHandlerTest, CoreAdminRequestStatusTest, TestModifyConfFiles, SolrCmdDistributorTest, DistributedExpandComponentTest, OverseerRolesTest, TestDocBasedVersionConstraints, FullSolrCloudDistribCmdsTest, CoreMergeIndexesAdminHandlerTest, SimplePostToolTest, HdfsLockFactoryTest, SSLMigrationTest, TermVectorComponentDistributedTest, SuggesterTSTTest, TestSweetSpotSimilarityFactory, MoreLikeThisHandlerTest, TestSuggestSpellingConverter, CSVRequestHandlerTest, TestRecovery, TestSolrXml, HdfsBasicDistributedZkTest, TestRemoteStreaming, DistribCursorPagingTest, QueryEqualityTest, TestNumberUtils, HdfsSyncSliceTest, TestSolrXmlPersistor, TestNoOpRegenerator, AutoCommitTest, TestPhraseSuggestions, UpdateParamsTest, TestShortCircuitedRequests, TestMergePolicyConfig, OutputWriterTest, TestJoin, LeaderElectionTest, TestLazyCores, ShardRoutingTest, QueryElevationComponentTest, HardAutoCommitTest, DocumentAnalysisRequestHandlerTest, HdfsCollectionsAPIDistributedZkTest, OverseerStatusTest, TestLuceneMatchVersion, AddSchemaFieldsUpdateProcessorFactoryTest, TestStressVersions, TestArbitraryIndexDir, TestSolrJ, RemoteQueryErrorTest, TestBlendedInfixSuggestions, TestLMJelinekMercerSimilarityFactory, TestBadConfig, ExpressionTest, TestFaceting, TestMultiCoreConfBootstrap, SuggesterFSTTest, PrimUtilsTest, AnalysisAfterCoreReloadTest, TestCoreContainer, PeerSyncTest, TestConfigSets, TestCloudManagedSchema, PingRequestHandlerTest, TestSerializedLuceneMatchVersion, TestFileDictionaryLookup, ShowFileRequestHandlerTest, CopyFieldTest, TestSolrIndexConfig, HdfsRecoveryZkTest, MigrateRouteKeyTest, TestSolrQueryParser, XmlUpdateRequestHandlerTest, UniqFieldsUpdateProcessorFactoryTest, ChaosMonkeyNothingIsSafeTest, TestSolrQueryParserResource, TestFuzzyAnalyzedSuggestions, SolrXmlInZkTest, NotRequiredUniqueKeyTest, TestRandomMergePolicy, StatsComponentTest, TestSimpleQParserPlugin, HdfsUnloadDistributedZkTest, FieldFacetExtrasTest, TestComplexPhraseQParserPlugin, TestShardHandlerFactory, TestSort, TestWordDelimiterFilterFactory, DistribDocExpirationUpdateProcessorTest, LoggingHandlerTest, BasicDistributedZkTest, AnalysisErrorHandlingTest, ZkSolrClientTest, LeaderElectionIntegrationTest, SchemaVersionSpecificBehaviorTest, BlockDirectoryTest, TestManagedResourceStorage, SolrRequestParserTest, TestOverriddenPrefixQueryForCustomFieldType, RequiredFieldsTest, TestDistribDocBasedVersion, LukeRequestHandlerTest, URLClassifyProcessorTest, TestCollationFieldDocValues, TestSearchPerf, TestSearcherReuse, CoreAdminHandlerTest, TestLFUCache, TestUpdate, DebugComponentTest, TermsComponentTest, ClusterStateUpdateTest, SynonymTokenizerTest, TestSolr4Spatial, TestReversedWildcardFilterFactory, IndexBasedSpellCheckerTest, TestCharFilters, RecoveryZkTest, DeleteReplicaTest, RequestHandlersTest, OpenCloseCoreStressTest, TestDistributedGrouping, TestDynamicFieldCollectionResource, AliasIntegrationTest, ConnectionManagerTest, CacheHeaderTest, TestStressLucene, TestReplicationHandler, PrimitiveFieldTypeTest, TestExtendedDismaxParser, TestFieldResource, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest, OverseerTest]
   [junit4] Completed in 65.84s, 9 tests, 1 error <<< FAILURES!

[...truncated 588 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:467: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:447: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:45: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:37: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:496: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1281: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:907: There were test failures: 397 suites, 1674 tests, 1 error, 42 ignored (21 assumptions)

Total time: 96 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_51 -XX:-UseCompressedOops -XX:+UseParallelGC -XX:-UseSuperWord
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message