lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-master - Build # 1621 - Unstable
Date Wed, 22 Aug 2018 12:06:10 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1621/

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded

Error Message:
ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@3115fd27, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@6d945ba2] expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: ComputePlanAction should have computed exactly 1 operation, but was: [org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@3115fd27, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@6d945ba2] expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([F81072539C74B440:9DD324243ED71C43]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeAdded(TestComputePlanAction.java:314)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13079 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction
   [junit4]   2> 301002 INFO  (SUITE-TestComputePlanAction-seed#[F81072539C74B440]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestComputePlanAction_F81072539C74B440-001/init-core-data-001
   [junit4]   2> 301003 WARN  (SUITE-TestComputePlanAction-seed#[F81072539C74B440]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=55 numCloses=55
   [junit4]   2> 301003 INFO  (SUITE-TestComputePlanAction-seed#[F81072539C74B440]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 301005 INFO  (SUITE-TestComputePlanAction-seed#[F81072539C74B440]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 301031 DEBUG (SUITE-TestComputePlanAction-seed#[F81072539C74B440]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10002_solr
   [junit4]   2> 301032 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 301032 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 301035 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 301035 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 301040 INFO  (SUITE-TestComputePlanAction-seed#[F81072539C74B440]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 301042 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10002_solr]
   [junit4]   2> 301048 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301048 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 301055 INFO  (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLost
   [junit4]   2> 301055 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 301055 DEBUG (ScheduledTrigger-862-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301055 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 301056 INFO  (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 301056 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 301056 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301056 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 301057 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 301057 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 301057 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 301057 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 301057 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 301057 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 301058 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 301058 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 301058 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 301058 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 301059 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10002_solr]
   [junit4]   2> 301059 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301059 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 301059 DEBUG (ScheduledTrigger-867-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301080 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301100 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301120 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301140 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301161 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301181 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301184 DEBUG (simCloudManagerPool-866-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 301184 DEBUG (simCloudManagerPool-866-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 301185 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 301185 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301185 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 301188 DEBUG (simCloudManagerPool-866-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 301188 DEBUG (simCloudManagerPool-866-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 301190 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 301191 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301191 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 301201 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301221 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301241 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301262 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301282 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301293 DEBUG (simCloudManagerPool-866-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 301293 DEBUG (simCloudManagerPool-866-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 301294 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 301295 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
   [junit4]   2> 301295 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301295 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 301295 DEBUG (ScheduledTrigger-867-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301296 DEBUG (simCloudManagerPool-866-thread-4) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeLost
   [junit4]   2> 301302 DEBUG (ScheduledTrigger-867-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 301310 DEBUG (simCloudManagerPool-866-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 301315 DEBUG (simCloudManagerPool-866-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 301316 DEBUG (ScheduledTrigger-867-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301322 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 301331 INFO  (simCloudManagerPool-866-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2227603237 sessionWrapper.createTime 2227602414851103, this.sessionWrapper.createTime 2227602414851103 
   [junit4]   2> 301332 DEBUG (simCloudManagerPool-866-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 301336 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301342 DEBUG (ScheduledTrigger-867-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 301343 DEBUG (simCloudManagerPool-866-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for testNodeLost / shard1
   [junit4]   2> 301343 DEBUG (simCloudManagerPool-866-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider Running leader election for testNodeLost / shard1
   [junit4]   2> 301346 DEBUG (simCloudManagerPool-866-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard1: core_node1
   [junit4]   2> 301354 DEBUG (simCloudManagerPool-866-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeLost / shard1
   [junit4]   2> 301354 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction -- cluster state: znodeVersion: 6
   [junit4]   2> live nodes:[127.0.0.1:10003_solr, 127.0.0.1:10002_solr]
   [junit4]   2> collections:{testNodeLost=DocCollection(testNodeLost//clusterstate.json/5)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testNodeLost_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testNodeLost_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}}
   [junit4]   2> 301356 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for testNodeLost / shard1
   [junit4]   2> 301356 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301356 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10003_solr
   [junit4]   2> 301357 INFO  (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction Stopped_node : 127.0.0.1:10003_solr
   [junit4]   2> 301363 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 301363 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10003_solr
   [junit4]   2> 301373 DEBUG (simCloudManagerPool-866-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider Running leader election for testNodeLost / shard1
   [junit4]   2> 301374 INFO  (simCloudManagerPool-866-thread-8) [    ] o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 3619ms
   [junit4]   2> 301376 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301383 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 301397 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301403 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 301417 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301423 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 301437 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301443 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 301446 DEBUG (simCloudManagerPool-866-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeLost / shard1: core_node2
   [junit4]   2> 301457 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301464 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 301480 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301484 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 301500 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 301501 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10003_solr]
   [junit4]   2> 301501 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"7e9fe9581d9efT2vgvnfdvdihacvyzhrezxilhu",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":2227604476254703,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2227604476254703],
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}}
   [junit4]   2> 301501 DEBUG (ScheduledTrigger-867-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance, node_lost_trigger]
   [junit4]   2> 301515 DEBUG (simCloudManagerPool-866-thread-9) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 301526 DEBUG (simCloudManagerPool-866-thread-9) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 301527 DEBUG (simCloudManagerPool-866-thread-9) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 301528 INFO  (simCloudManagerPool-866-thread-9) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2227613054 sessionWrapper.createTime 2227613017894553, this.sessionWrapper.createTime 2227613017894553 
   [junit4]   2> 301528 DEBUG (simCloudManagerPool-866-thread-9) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 301548 DEBUG (simCloudManagerPool-866-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for .system / shard1
   [junit4]   2> 301561 DEBUG (simCloudManagerPool-866-thread-11) [    ] o.a.s.c.a.s.SimClusterStateProvider Running leader election for .system / shard1
   [junit4]   2> 301562 DEBUG (simCloudManagerPool-866-thread-11) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1: core_node1
   [junit4]   2> 301570 DEBUG (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"7e9fe9581d9efT2vgvnfdvdihacvyzhrezxilhu",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":2227604476254703,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2227604476254703],
   [junit4]   2>     "_enqueue_time_":2227611740314953,
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}}
   [junit4]   2> 301570 DEBUG (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"7e9fe9581d9efT2vgvnfdvdihacvyzhrezxilhu",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":2227604476254703,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2227604476254703],
   [junit4]   2>     "_enqueue_time_":2227611740314953,
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 301570 DEBUG (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 301578 DEBUG (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 301582 INFO  (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeLost&targetNode=127.0.0.1:10004_solr&inPlaceMove=true&replica=core_node1
   [junit4]   2> 301582 INFO  (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2227615792 sessionWrapper.createTime 2227615556039353, this.sessionWrapper.createTime 2227615556039353 
   [junit4]   2> 301583 DEBUG (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 301601 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for .system / shard1
   [junit4]   2> 301617 INFO  (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction -------------_ FINAL STATE --------------
   [junit4]   2> 301617 INFO  (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction * Node values: {
   [junit4]   2>   "127.0.0.1:10004_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":1023,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10004_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10004,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10002_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":999,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10002_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10002,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10003_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":1023,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10003_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10003,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
   [junit4]   2> 301618 INFO  (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction * Live nodes: [127.0.0.1:10002_solr]
   [junit4]   2> 301618 INFO  (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction * Collection .system state: DocCollection(.system//clusterstate.json/14)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"1",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{"core_node1":{
   [junit4]   2>           "core":".system_shard1_replica_n1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":3,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":11740,
   [junit4]   2>           "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":1.0933727025985718E-5,
   [junit4]   2>           "SEARCHER.searcher.numDocs":3}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 301618 INFO  (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction * Collection testNodeLost state: DocCollection(testNodeLost//clusterstate.json/14)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testNodeLost_shard1_replica_n1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testNodeLost_shard1_replica_n2",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 301618 INFO  (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLost
   [junit4]   2> 301619 INFO  (TEST-TestComputePlanAction.testNodeLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		1
   [junit4]   2> ## Empty nodes:	0
   [junit4]   2> ## Dead nodes:		2
   [junit4]   2> ##		127.0.0.1:10003_solr
   [junit4]   2> ##		127.0.0.1:10004_solr
   [junit4]   2> ## Collections:	[.system, testNodeLost]
   [junit4]   2> ## Max replicas per node:	1
   [junit4]   2> ## Min replicas per node:	1
   [junit4]   2> ## Total replicas:		1
   [junit4]   2> ## * testNodeLost		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- CREATE             1
   [junit4]   2> ##		- autoscaling        3
   [junit4]   2> ##		- update             3
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_lost_trigger
   [junit4]   2> ##		- AFTER_ACTION     1
   [junit4]   2> ##		- BEFORE_ACTION     1
   [junit4]   2> ##		- STARTED         1
   [junit4]   2> 
   [junit4]   2> 301622 DEBUG (simCloudManagerPool-866-thread-14) [    ] o.a.s.c.a.s.SimClusterStateProvider Running leader election for .system / shard1
   [junit4]   2> 301630 WARN  (simCloudManagerPool-866-thread-14) [    ] o.a.s.c.a.s.SimClusterStateProvider Can't find any active replicas for .system / shard1: shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":3,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":11740,
   [junit4]   2>       "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":1.0933727025985718E-5,
   [junit4]   2>       "SEARCHER.searcher.numDocs":3}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 301630 DEBUG (simCloudManagerPool-866-thread-15) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":3,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":11740,
   [junit4]   2>       "node_name":"127.0.0.1:10004_solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":1.0933727025985718E-5,
   [junit4]   2>       "SEARCHER.searcher.numDocs":3}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 301630 DEBUG (simCloudManagerPool-866-thread-14) [    ] o.a.s.c.a.s.SimClusterStateProvider -- liveNodes: [127.0.0.1:10002_solr]
   [junit4]   2> 301630 DEBUG (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 301631 DEBUG (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
   [junit4]   2> 301631 DEBUG (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
   [junit4]   2> 301631 DEBUG (AutoscalingActionExecutor-868-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 3040 ms for event id=7e9fe9581d9efT2vgvnfdvdihacvyzhrezxilhu
   [junit4]   2> 301634 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeWithMultipleReplicasLost
   [junit4]   2> 301634 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 301635 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10004_solr
   [junit4]   2> 301635 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10003_solr
   [junit4]   2> 301635 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 301635 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 301635 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 301635 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 301635 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 301635 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 301635 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 301636 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 301646 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 301646 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 301647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
   [junit4]   2> 301647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 301649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10002_solr]
   [junit4]   2> 301655 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301656 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 301656 DEBUG (ScheduledTrigger-870-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301676 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301697 DEBUG (ScheduledTrigger-870-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301718 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301738 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301747 DEBUG (simCloudManagerPool-869-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 301747 DEBUG (simCloudManagerPool-869-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 301747 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 301748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301748 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 301748 DEBUG (simCloudManagerPool-869-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 301748 DEBUG (simCloudManagerPool-869-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
   [junit4]   2> 301749 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 301749 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301749 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 301761 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301781 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301801 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301822 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301842 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 301851 DEBUG (simCloudManagerPool-869-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 301851 DEBUG (simCloudManagerPool-869-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
   [junit4]   2> 301852 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
   [junit4]   2> 301853 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10006_solr, 127.0.0.1:10005_solr, 127.0.0.1:10002_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 301853 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 301853 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
   [junit4]   2> 301853 DEBUG (ScheduledTrigger-870-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 301853 DEBUG (simCloudManagerPool-869-thread-4) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeWithMultipleReplicasLost
   [junit4]   2> 301862 DEBUG (ScheduledTrigger-870-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 301864 DEBUG (simCloudManagerPool-869-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 301864 DEBUG (simCloudManagerPool-869-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 301873 DEBUG (ScheduledTrigger-870-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 301882 DEBUG (ScheduledTrigger-870-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 301883 INFO  (simCloudManagerPool-869-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2227630827 sessionWrapper.createTime 2227629892675903, this.sessionWrapper.createTime 2227629892675903 
   [junit4]   2> 301883 DEBUG (simCloudManagerPool-869-thread-4) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 301894 DEBUG (ScheduledTrigger-870-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 301902 DEBUG (simCloudManagerPool-869-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no replicas in testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 301902 DEBUG (simCloudManagerPool-869-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 301902 DEBUG (simCloudManagerPool-869-thread-11) [    ] o.a.s.c.a.s.SimClusterStateProvider Running leader election for testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 301903 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 301903 DEBUG (simCloudManagerPool-869-thread-11) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeWithMultipleReplicasLost / shard1: core_node1
   [junit4]   2> 301913 DEBUG (simCloudManagerPool-869-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no replicas in testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 301913 DEBUG (simCloudManagerPool-869-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 301914 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 301923 DEBUG (ScheduledTrigger-870-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 301924 DEBUG (simCloudManagerPool-869-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no replicas in testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 301924 DEBUG (simCloudManagerPool-869-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 301935 DEBUG (simCloudManagerPool-869-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 301935 DEBUG (ScheduledTrigger-870-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 301935 DEBUG (simCloudManagerPool-869-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 301935 DEBUG (simCloudManagerPool-869-thread-12) [    ] o.a.s.c.a.s.SimClusterStateProvider Running leader election for testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 301936 DEBUG (simCloudManagerPool-869-thread-12) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeWithMultipleReplicasLost / shard2: core_node4
   [junit4]   2> 301944 DEBUG (ScheduledTrigger-870-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 301946 DEBUG (simCloudManagerPool-869-thread-9) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 301946 DEBUG (simCloudManagerPool-869-thread-9) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 301955 DEBUG (ScheduledTrigger-870-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 301958 DEBUG (simCloudManagerPool-869-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 301958 DEBUG (simCloudManagerPool-869-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 301958 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction -- cluster state: znodeVersion: 25
   [junit4]   2> live nodes:[127.0.0.1:10002_solr, 127.0.0.1:10006_solr, 127.0.0.1:10005_solr, 127.0.0.1:10007_solr]
   [junit4]   2> collections:{testNodeWithMultipleReplicasLost=DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/24)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard2":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10006_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active"},
   [junit4]   2>     "shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10007_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard1_replica_n3",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10006_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active"}}}}
   [junit4]   2> 301959 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeWithMultipleReplicasLost / shard2
   [junit4]   2> 301959 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeWithMultipleReplicasLost / shard1
   [junit4]   2> 301964 DEBUG (ScheduledTrigger-870-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 301964 DEBUG (ScheduledTrigger-870-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10006_solr
   [junit4]   2> 301976 DEBUG (ScheduledTrigger-870-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 301976 DEBUG (ScheduledTrigger-870-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10006_solr
   [junit4]   2> 301985 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 301996 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 301996 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10006_solr]
   [junit4]   2> 301996 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"7ea05cc126d51T2vgvnfdvdihacvyzhrezxilhz",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":2227635456470353,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2227635456470353],
   [junit4]   2>     "nodeNames":["127.0.0.1:10006_solr"]}}
   [junit4]   2> 301997 DEBUG (ScheduledTrigger-870-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance, node_lost_trigger]
   [junit4]   2> 302000 DEBUG (simCloudManagerPool-869-thread-13) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 302011 DEBUG (simCloudManagerPool-869-thread-13) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 302012 DEBUG (simCloudManagerPool-869-thread-13) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 302013 INFO  (simCloudManagerPool-869-thread-13) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2227637297 sessionWrapper.createTime 2227637252622303, this.sessionWrapper.createTime 2227637252622303 
   [junit4]   2> 302013 DEBUG (simCloudManagerPool-869-thread-13) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 302024 DEBUG (simCloudManagerPool-869-thread-14) [    ] o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for .system / shard1
   [junit4]   2> 302024 DEBUG (simCloudManagerPool-869-thread-15) [    ] o.a.s.c.a.s.SimClusterStateProvider Running leader election for .system / shard1
   [junit4]   2> 302025 DEBUG (simCloudManagerPool-869-thread-15) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1: core_node2
   [junit4]   2> 302029 DEBUG (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"7ea05cc126d51T2vgvnfdvdihacvyzhrezxilhz",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":2227635456470353,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2227635456470353],
   [junit4]   2>     "_enqueue_time_":2227636503817953,
   [junit4]   2>     "nodeNames":["127.0.0.1:10006_solr"]}}
   [junit4]   2> 302031 DEBUG (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"7ea05cc126d51T2vgvnfdvdihacvyzhrezxilhz",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":2227635456470353,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[2227635456470353],
   [junit4]   2>     "_enqueue_time_":2227636503817953,
   [junit4]   2>     "nodeNames":["127.0.0.1:10006_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 302031 DEBUG (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 302039 DEBUG (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 302050 INFO  (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10007_solr&inPlaceMove=true&replica=core_node5
   [junit4]   2> 302051 INFO  (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10002_solr&inPlaceMove=true&replica=core_node3
   [junit4]   2> 302051 INFO  (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2227639227 sessionWrapper.createTime 2227638733080153, this.sessionWrapper.createTime 2227638733080153 
   [junit4]   2> 302051 DEBUG (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 302055 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10007_solr&inPlaceMove=true&replica=core_node5
   [junit4]   2> 302055 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10002_solr&inPlaceMove=true&replica=core_node3
   [junit4]   2> 302055 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction -------------_ FINAL STATE --------------
   [junit4]   2> 302055 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction * Node values: {
   [junit4]   2>   "127.0.0.1:10006_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":2,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":1022,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10006_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10006,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10005_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":2,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":1022,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10005_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10005,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10002_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":1,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":999,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10002_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10002,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
   [junit4]   2>   "127.0.0.1:10007_solr":{
   [junit4]   2>     "sysprop.java.vendor":"Oracle Corporation",
   [junit4]   2>     "cores":2,
   [junit4]   2>     "host":"127.0.0.1",
   [junit4]   2>     "freedisk":1022,
   [junit4]   2>     "sysprop.java.version":"1.8.0_172",
   [junit4]   2>     "ip_1":"127",
   [junit4]   2>     "ip_3":"0",
   [junit4]   2>     "ip_2":"0",
   [junit4]   2>     "ip_4":"1",
   [junit4]   2>     "node":"127.0.0.1:10007_solr",
   [junit4]   2>     "metrics:solr.jvm:buffers.direct.Count":0,
   [junit4]   2>     "port":10007,
   [junit4]   2>     "heapUsage":123450000,
   [junit4]   2>     "sysLoadAvg":1.0,
   [junit4]   2>     "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
   [junit4]   2> 302055 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction * Live nodes: [127.0.0.1:10005_solr, 127.0.0.1:10002_solr, 127.0.0.1:10007_solr]
   [junit4]   2> 302056 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction * Collection .system state: DocCollection(.system//clusterstate.json/32)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"1",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "replicas":{"core_node2":{
   [junit4]   2>           "core":".system_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":2,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":11240,
   [junit4]   2>           "node_name":"127.0.0.1:10007_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":1.0468065738677979E-5,
   [junit4]   2>           "SEARCHER.searcher.numDocs":2}},
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 302056 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.TestComputePlanAction * Collection testNodeWithMultipleReplicasLost state: DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/32)={
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard2":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10002_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node5":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10006_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"active"},
   [junit4]   2>     "shard1":{
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10007_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0},
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeWithMultipleReplicasLost_shard1_replica_n3",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":10240,
   [junit4]   2>           "node_name":"127.0.0.1:10006_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>           "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"active"}}}
   [junit4]   2> 302056 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeWithMultipleReplicasLost
   [junit4]   2> 302057 INFO  (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes:		3
   [junit4]   2> ## Empty nodes:	0
   [junit4]   2> ## Dead nodes:		1
   [junit4]   2> ##		127.0.0.1:10006_solr
   [junit4]   2> ## Collections:	[.system, testNodeWithMultipleReplicasLost]
   [junit4]   2> ## Max replicas per node:	2
   [junit4]   2> ## Min replicas per node:	1
   [junit4]   2> ## Total replicas:		5
   [junit4]   2> ## * .system		1
   [junit4]   2> ##		- active           1
   [junit4]   2> ## * testNodeWithMultipleReplicasLost		4
   [junit4]   2> ##		- active           4
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ##		- CREATE             1
   [junit4]   2> ##		- autoscaling        3
   [junit4]   2> ##		- update             2
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_lost_trigger
   [junit4]   2> ##		- BEFORE_ACTION     1
   [junit4]   2> ##		- STARTED         1
   [junit4]   2> 
   [junit4]   2> 302066 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAdded
   [junit4]   2> 302066 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
   [junit4]   2> 302067 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
   [junit4]   2> 302067 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10006_solr
   [junit4]   2> 302067 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 302067 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10007_solr
   [junit4]   2> 302067 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 302067 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
   [junit4]   2> 302067 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10005_solr
   [junit4]   2> 302067 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10002_solr
   [junit4]   2> 302067 INFO  (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 302067 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 302067 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 302067 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 302069 DEBUG (simCloudManagerPool-869-thread-17) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 302069 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 302069 WARN  (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.lang.InterruptedException
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:792) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:768) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:315) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630) ~[test/:?]
   [junit4]   2> 	... 10 more
   [junit4]   2> 302070 DEBUG (simCloudManagerPool-869-thread-18) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
   [junit4]   2> 302090 DEBUG (simCloudManagerPool-869-thread-19) [    ] o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for .system / shard1
   [junit4]   2> 302091 DEBUG (simCloudManagerPool-869-thread-20) [    ] o.a.s.c.a.s.SimClusterStateProvider Running leader election for .system / shard1
   [junit4]   2> 302091 DEBUG (simCloudManagerPool-869-thread-20) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for .system / shard1: core_node3
   [junit4]   2> 303292 WARN  (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Unhandled exception executing actions
   [junit4]   2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_lost_trigger has been closed.
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:548) ~[java/:?]
   [junit4]   2> 	at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:318) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 303293 DEBUG (AutoscalingActionExecutor-871-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 63179 ms for event id=7ea05cc126d51T2vgvnfdvdihacvyzhrezxilhz
   [junit4]   2> 303296 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 303296 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 303296 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[F81072539C74B440]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 303299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 303299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
   [junit4]   2> 303299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion -1
   [junit4]   2> 303299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
   [junit4]   2> 303302 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10005_solr]
   [junit4]   2> 303302 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303302 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
   [junit4]   2> 303305 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303326 DEBUG (ScheduledTrigger-873-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303346 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303367 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303387 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303403 DEBUG (simCloudManagerPool-872-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303403 DEBUG (simCloudManagerPool-872-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
   [junit4]   2> 303405 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
   [junit4]   2> 303405 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303405 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
   [junit4]   2> 303406 DEBUG (simCloudManagerPool-872-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303406 DEBUG (simCloudManagerPool-872-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
   [junit4]   2> 303407 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
   [junit4]   2> 303407 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 303407 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
   [junit4]   2> 303407 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303428 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303448 DEBUG (ScheduledTrigger-873-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303469 DEBUG (ScheduledTrigger-873-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303489 DEBUG (ScheduledTrigger-873-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303509 DEBUG (ScheduledTrigger-873-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303518 DEBUG (simCloudManagerPool-872-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303519 DEBUG (simCloudManagerPool-872-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
   [junit4]   2> 303519 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
   [junit4]   2> 303520 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10005_solr]
   [junit4]   2> 303520 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction$AssertingTriggerAction}], enabled=true}
   [junit4]   2> 303520 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
   [junit4]   2> 303520 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 303520 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 303528 DEBUG (simCloudManagerPool-872-thread-4) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 303528 DEBUG (simCloudManagerPool-872-thread-4) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
   [junit4]   2> 303528 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
   [junit4]   2> 303529 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
   [junit4]   2> 303529 DEBUG (ScheduledTrigger-873-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303530 DEBUG (simCloudManagerPool-872-thread-5) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeAdded
   [junit4]   2> 303542 DEBUG (ScheduledTrigger-873-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 303542 DEBUG (ScheduledTrigger-873-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 303543 WARN  (simCloudManagerPool-872-thread-5) [    ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 4 on collection testNodeAdded is higher than the number of Solr instances currently live or live and part of your createNodeSet(1). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 303543 DEBUG (simCloudManagerPool-872-thread-5) [    ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
   [junit4]   2> 303543 DEBUG (simCloudManagerPool-872-thread-5) [    ] o.a.s.c.s.c.a.PolicyHelper New session created 
   [junit4]   2> 303550 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303552 INFO  (simCloudManagerPool-872-thread-5) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2227714244 sessionWrapper.createTime 2227713837414203, this.sessionWrapper.createTime 2227713837414203 
   [junit4]   2> 303552 DEBUG (simCloudManagerPool-872-thread-5) [    ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
   [junit4]   2> 303562 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 303562 DEBUG (ScheduledTrigger-873-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 303564 DEBUG (simCloudManagerPool-872-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- submit leader election for testNodeAdded / shard1
   [junit4]   2> 303567 DEBUG (simCloudManagerPool-872-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider Running leader election for testNodeAdded / shard1
   [junit4]   2> 303567 DEBUG (simCloudManagerPool-872-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeAdded / shard1: core_node1
   [junit4]   2> 303570 DEBUG (ScheduledTrigger-873-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303577 DEBUG (simCloudManagerPool-872-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeAdded / shard1
   [junit4]   2> 303583 DEBUG (ScheduledTrigger-873-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 303583 DEBUG (ScheduledTrigger-873-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 303588 DEBUG (simCloudManagerPool-872-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeAdded / shard1
   [junit4]   2> 303590 DEBUG (ScheduledTrigger-873-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 303599 DEBUG (simCloudManagerPool-872-thread-9) [    ] o.a.s.c.a.s.SimClusterStateProvider -- already has leader testNodeAdded / shard1
   [junit4]   2> 303602 DEBUG (simCloudManagerPool-872-thre

[...truncated too long message...]



ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null1989748819
     [copy] Copying 238 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null1989748819
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null1989748819

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 49ms :: artifacts dl 5ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 266 minutes 10 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$34.hasMatch(FilePath.java:2678)
	at hudson.FilePath$34.invoke(FilePath.java:2557)
	at hudson.FilePath$34.invoke(FilePath.java:2547)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1036)
		at hudson.FilePath.act(FilePath.java:1025)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1819)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1038)
	at hudson.FilePath.act(FilePath.java:1025)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

Mime
View raw message