lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 217 - Still Unstable
Date Fri, 02 Dec 2016 21:08:11 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/217/

4 tests failed.
FAILED:  org.apache.lucene.search.TestFuzzyQuery.testRandom

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([45F89C17A5E63F03]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestFuzzyQuery

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([45F89C17A5E63F03]:0)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/27)={   "replicationFactor":"4",   "shards":{     "shard1":{       "range":"80000000-bfffffff",       "state":"active",       "replicas":{}},     "shard2":{       "range":"c0000000-ffffffff",       "state":"active",       "replicas":{}},     "shard3":{       "range":"0-3fffffff",       "state":"active",       "replicas":{}},     "shard4":{       "range":"40000000-7fffffff",       "state":"active",       "replicas":{}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"5",   "autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/27)={
  "replicationFactor":"4",
  "shards":{
    "shard1":{
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{}},
    "shard2":{
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{}},
    "shard3":{
      "range":"0-3fffffff",
      "state":"active",
      "replicas":{}},
    "shard4":{
      "range":"40000000-7fffffff",
      "state":"active",
      "replicas":{}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"5",
  "autoAddReplicas":"false"}
	at __randomizedtesting.SeedInfo.seed([C8F88680C76DA882:808DF234C15E8717]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:237)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets

Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2

Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
	at __randomizedtesting.SeedInfo.seed([C8F88680C76DA882:E586C9DA3049128E]:0)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 1709 lines...]
   [junit4] Suite: org.apache.lucene.search.TestFuzzyQuery
   [junit4]   2> Δεκ 02, 2016 2:38:07 ΜΜ com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: org.apache.lucene.search.TestFuzzyQuery
   [junit4]   2>    1) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
   [junit4]   2>         at java.lang.Thread.sleep(Native Method)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
   [junit4]   2>    2) Thread[id=2206, name=TEST-TestFuzzyQuery.testRandom-seed#[45F89C17A5E63F03], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
   [junit4]   2>         at java.util.WeakHashMap.getTable(WeakHashMap.java:350)
   [junit4]   2>         at java.util.WeakHashMap.get(WeakHashMap.java:397)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedContext.getPerThread(RandomizedContext.java:319)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedContext.getRandomness(RandomizedContext.java:92)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedContext.getRandom(RandomizedContext.java:114)
   [junit4]   2>         at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
   [junit4]   2>         at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
   [junit4]   2>         at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=1, name=main, state=WAITING, group=main]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1249)
   [junit4]   2>         at java.lang.Thread.join(Thread.java:1323)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:243)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:354)
   [junit4]   2>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
   [junit4]   2>    4) Thread[id=2205, name=SUITE-TestFuzzyQuery-seed#[45F89C17A5E63F03], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
   [junit4]   2>         at java.lang.Thread.getStackTrace(Thread.java:1556)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:690)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:687)
   [junit4]   2>         at java.security.AccessController.doPrivileged(Native Method)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:687)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:703)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:683)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestFuzzyQuery -Dtests.method=testRandom -Dtests.seed=45F89C17A5E63F03 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=el -Dtests.timezone=America/Argentina/Ushuaia -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   7200s J1 | TestFuzzyQuery.testRandom <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([45F89C17A5E63F03]:0)
   [junit4]   2> Δεκ 02, 2016 2:38:07 ΜΜ com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> Δεκ 02, 2016 2:38:27 ΜΜ com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.search.TestFuzzyQuery: 
   [junit4]   2>    1) Thread[id=2206, name=TEST-TestFuzzyQuery.testRandom-seed#[45F89C17A5E63F03], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
   [junit4]   2>         at java.util.WeakHashMap.getTable(WeakHashMap.java:350)
   [junit4]   2>         at java.util.WeakHashMap.getEntry(WeakHashMap.java:427)
   [junit4]   2>         at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
   [junit4]   2>         at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
   [junit4]   2>         at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
   [junit4]   2>         at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Δεκ 02, 2016 2:38:27 ΜΜ com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=2206, name=TEST-TestFuzzyQuery.testRandom-seed#[45F89C17A5E63F03], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
   [junit4]   2> Δεκ 02, 2016 2:38:31 ΜΜ com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=2206, name=TEST-TestFuzzyQuery.testRandom-seed#[45F89C17A5E63F03], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
   [junit4]   2>         at java.lang.Object.hashCode(Native Method)
   [junit4]   2>         at java.util.WeakHashMap.hash(WeakHashMap.java:298)
   [junit4]   2>         at java.util.WeakHashMap.getEntry(WeakHashMap.java:426)
   [junit4]   2>         at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
   [junit4]   2>         at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
   [junit4]   2>         at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:503)
   [junit4]   2>         at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>         at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {field=Lucene50(blocksize=128)}, docValues:{}, maxPointsInLeafNode=1392, maxMBSortInHeap=7.608402405072461, sim=RandomSimilarity(queryNorm=false,coord=no): {field=DFR I(ne)LZ(0.3)}, locale=el, timezone=America/Argentina/Ushuaia
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=98407144,total=239599616
   [junit4]   2> NOTE: All tests run in this JVM: [TestWeakIdentityMap, TestFilterWeight, TestIndexWriterOnVMError, TestLSBRadixSorter, TestSleepingLockWrapper, TestDocValues, TestHighCompressionMode, TestDemo, TestMultiFields, TestAddIndexes, TestExitableDirectoryReader, TestSloppyMath, TestIndexingSequenceNumbers, TestMatchNoDocsQuery, TestMultiPhraseEnum, TestIsCurrent, TestSearch, TestFixedLengthBytesRefArray, TestSpanContainQuery, TestNumericUtils, TestStringHelper, TestBM25Similarity, TestBytesRefHash, TestSwappedIndexFiles, TestPolygon, TestReusableStringReader, TestNRTReaderCleanup, TestPointQueries, TestCompiledAutomaton, TestFuzzyQuery]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestFuzzyQuery -Dtests.seed=45F89C17A5E63F03 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=el -Dtests.timezone=America/Argentina/Ushuaia -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestFuzzyQuery (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([45F89C17A5E63F03]:0)
   [junit4] Completed [398/442 (1!)] on J1 in 7223.20s, 11 tests, 2 errors <<< FAILURES!

[...truncated 9701 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001/init-core-data-001
   [junit4]   2> 38572 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 38574 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001/tempDir-001
   [junit4]   2> 38574 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 38584 INFO  (Thread-47) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 38584 INFO  (Thread-47) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 38684 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:48373
   [junit4]   2> 38741 INFO  (jetty-launcher-36-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 38748 INFO  (jetty-launcher-36-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 38760 INFO  (jetty-launcher-36-thread-3) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 38769 INFO  (jetty-launcher-36-thread-4) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 38849 INFO  (jetty-launcher-36-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4f847553{/solr,null,AVAILABLE}
   [junit4]   2> 38851 INFO  (jetty-launcher-36-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@8776a32{HTTP/1.1,[http/1.1]}{127.0.0.1:43051}
   [junit4]   2> 38851 INFO  (jetty-launcher-36-thread-2) [    ] o.e.j.s.Server Started @41852ms
   [junit4]   2> 38851 INFO  (jetty-launcher-36-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43051}
   [junit4]   2> 38851 ERROR (jetty-launcher-36-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 38851 INFO  (jetty-launcher-36-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 6.4.0
   [junit4]   2> 38851 INFO  (jetty-launcher-36-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 38851 INFO  (jetty-launcher-36-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 38851 INFO  (jetty-launcher-36-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-12-02T19:31:29.455Z
   [junit4]   2> 38890 INFO  (jetty-launcher-36-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7e9b35de{/solr,null,AVAILABLE}
   [junit4]   2> 38890 INFO  (jetty-launcher-36-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4d2c5503{HTTP/1.1,[http/1.1]}{127.0.0.1:49074}
   [junit4]   2> 38890 INFO  (jetty-launcher-36-thread-1) [    ] o.e.j.s.Server Started @41891ms
   [junit4]   2> 38891 INFO  (jetty-launcher-36-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49074}
   [junit4]   2> 38891 ERROR (jetty-launcher-36-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 38891 INFO  (jetty-launcher-36-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 6.4.0
   [junit4]   2> 38891 INFO  (jetty-launcher-36-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 38891 INFO  (jetty-launcher-36-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 38891 INFO  (jetty-launcher-36-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-12-02T19:31:29.495Z
   [junit4]   2> 38933 INFO  (jetty-launcher-36-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36d9f226{/solr,null,AVAILABLE}
   [junit4]   2> 38933 INFO  (jetty-launcher-36-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@45044b92{HTTP/1.1,[http/1.1]}{127.0.0.1:51561}
   [junit4]   2> 38933 INFO  (jetty-launcher-36-thread-4) [    ] o.e.j.s.Server Started @41934ms
   [junit4]   2> 38933 INFO  (jetty-launcher-36-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51561}
   [junit4]   2> 38934 ERROR (jetty-launcher-36-thread-4) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 38934 INFO  (jetty-launcher-36-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 6.4.0
   [junit4]   2> 38934 INFO  (jetty-launcher-36-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 38934 INFO  (jetty-launcher-36-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 38934 INFO  (jetty-launcher-36-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-12-02T19:31:29.538Z
   [junit4]   2> 38954 INFO  (jetty-launcher-36-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@88031f6{/solr,null,AVAILABLE}
   [junit4]   2> 38954 INFO  (jetty-launcher-36-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@52c83ebd{HTTP/1.1,[http/1.1]}{127.0.0.1:44541}
   [junit4]   2> 38954 INFO  (jetty-launcher-36-thread-3) [    ] o.e.j.s.Server Started @41955ms
   [junit4]   2> 38954 INFO  (jetty-launcher-36-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44541}
   [junit4]   2> 38955 ERROR (jetty-launcher-36-thread-3) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 38955 INFO  (jetty-launcher-36-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 6.4.0
   [junit4]   2> 38955 INFO  (jetty-launcher-36-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 38955 INFO  (jetty-launcher-36-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 38955 INFO  (jetty-launcher-36-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-12-02T19:31:29.559Z
   [junit4]   2> 38993 INFO  (jetty-launcher-36-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 38994 INFO  (jetty-launcher-36-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 39021 INFO  (jetty-launcher-36-thread-2) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 39036 INFO  (jetty-launcher-36-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48373/solr
   [junit4]   2> 39057 INFO  (jetty-launcher-36-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 39072 INFO  (jetty-launcher-36-thread-4) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 39057 INFO  (jetty-launcher-36-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 39090 INFO  (jetty-launcher-36-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48373/solr
   [junit4]   2> 39105 INFO  (jetty-launcher-36-thread-3) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 39124 INFO  (jetty-launcher-36-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48373/solr
   [junit4]   2> 39136 INFO  (jetty-launcher-36-thread-1) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 39185 INFO  (jetty-launcher-36-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48373/solr
   [junit4]   2> 39272 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x158c10523320007, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 39777 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x158c10523320009, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 39848 INFO  (jetty-launcher-36-thread-2) [n:127.0.0.1:43051_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43051_solr
   [junit4]   2> 39849 INFO  (jetty-launcher-36-thread-4) [n:127.0.0.1:51561_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51561_solr
   [junit4]   2> 39849 INFO  (jetty-launcher-36-thread-2) [n:127.0.0.1:43051_solr    ] o.a.s.c.Overseer Overseer (id=97039619797942280-127.0.0.1:43051_solr-n_0000000000) starting
   [junit4]   2> 39860 INFO  (jetty-launcher-36-thread-1) [n:127.0.0.1:49074_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49074_solr
   [junit4]   2> 39888 INFO  (zkCallback-53-thread-1-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 39912 INFO  (zkCallback-56-thread-1-processing-n:127.0.0.1:49074_solr) [n:127.0.0.1:49074_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 39929 INFO  (zkCallback-55-thread-1-processing-n:127.0.0.1:51561_solr) [n:127.0.0.1:51561_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 39940 INFO  (jetty-launcher-36-thread-3) [n:127.0.0.1:44541_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 39944 INFO  (jetty-launcher-36-thread-3) [n:127.0.0.1:44541_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44541_solr
   [junit4]   2> 39945 INFO  (zkCallback-53-thread-1-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 39945 INFO  (zkCallback-55-thread-1-processing-n:127.0.0.1:51561_solr) [n:127.0.0.1:51561_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 39945 INFO  (zkCallback-56-thread-1-processing-n:127.0.0.1:49074_solr) [n:127.0.0.1:49074_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 39966 INFO  (jetty-launcher-36-thread-1) [n:127.0.0.1:49074_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001/tempDir-001/node1/.
   [junit4]   2> 40117 INFO  (jetty-launcher-36-thread-2) [n:127.0.0.1:43051_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43051_solr
   [junit4]   2> 40118 INFO  (zkCallback-56-thread-1-processing-n:127.0.0.1:49074_solr) [n:127.0.0.1:49074_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 40118 INFO  (zkCallback-53-thread-1-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 40118 INFO  (zkCallback-55-thread-1-processing-n:127.0.0.1:51561_solr) [n:127.0.0.1:51561_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 40279 INFO  (zkCallback-57-thread-1-processing-n:127.0.0.1:44541_solr) [n:127.0.0.1:44541_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 40378 INFO  (jetty-launcher-36-thread-3) [n:127.0.0.1:44541_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001/tempDir-001/node3/.
   [junit4]   2> 40388 INFO  (jetty-launcher-36-thread-2) [n:127.0.0.1:43051_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001/tempDir-001/node2/.
   [junit4]   2> 40456 INFO  (jetty-launcher-36-thread-4) [n:127.0.0.1:51561_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001/tempDir-001/node4/.
   [junit4]   2> 40527 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 40530 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:48373/solr ready
   [junit4]   2> 41811 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 42598 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 42776 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 42795 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 42926 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 42965 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_49917_hdfs____9zsqas/webapp
   [junit4]   2> 43917 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49917
   [junit4]   2> 45122 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 45126 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 45135 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_54829_datanode____.1904gp/webapp
   [junit4]   2> 45563 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54829
   [junit4]   2> 46235 WARN  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 46237 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 46255 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_39051_datanode____a688st/webapp
   [junit4]   2> 47200 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39051
   [junit4]   2> 49028 INFO  (IPC Server handler 8 on 47085) [    ] BlockStateChange BLOCK* processReport: from storage DS-1d0f7b9b-82a2-4e70-b0a1-9eda63ceb04e node DatanodeRegistration(127.0.0.1:53746, datanodeUuid=68dab75c-e896-408f-bb83-94caf7ae844a, infoPort=56861, infoSecurePort=0, ipcPort=60912, storageInfo=lv=-56;cid=testClusterID;nsid=1684498258;c=0), blocks: 0, hasStaleStorage: true, processing time: 2 msecs
   [junit4]   2> 49029 INFO  (IPC Server handler 8 on 47085) [    ] BlockStateChange BLOCK* processReport: from storage DS-4a43231a-7ebb-4296-916e-b30d86c5a665 node DatanodeRegistration(127.0.0.1:53746, datanodeUuid=68dab75c-e896-408f-bb83-94caf7ae844a, infoPort=56861, infoSecurePort=0, ipcPort=60912, storageInfo=lv=-56;cid=testClusterID;nsid=1684498258;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 49038 INFO  (IPC Server handler 6 on 47085) [    ] BlockStateChange BLOCK* processReport: from storage DS-d1730711-1c4c-4fd5-b457-f602411d1ff2 node DatanodeRegistration(127.0.0.1:41582, datanodeUuid=21058415-1975-429a-8cc1-7c68cde00022, infoPort=48829, infoSecurePort=0, ipcPort=42086, storageInfo=lv=-56;cid=testClusterID;nsid=1684498258;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 49038 INFO  (IPC Server handler 6 on 47085) [    ] BlockStateChange BLOCK* processReport: from storage DS-3814a25d-11a7-41ce-a41f-fc171e8b0447 node DatanodeRegistration(127.0.0.1:41582, datanodeUuid=21058415-1975-429a-8cc1-7c68cde00022, infoPort=48829, infoSecurePort=0, ipcPort=42086, storageInfo=lv=-56;cid=testClusterID;nsid=1684498258;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 49185 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.testCreateNodeSet-seed#[C8F88680C76DA882]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCreateNodeSet
   [junit4]   2> 49190 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.testCreateNodeSet-seed#[C8F88680C76DA882]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 49201 INFO  (qtp141464611-258) [n:127.0.0.1:51561_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=nodeset_collection&action=CREATE&numShards=2&createNodeSet=http://127.0.0.1:51561/solr,http://127.0.0.1:49074/solr&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 49205 INFO  (OverseerThreadFactory-91-thread-1-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.CreateCollectionCmd Create collection nodeset_collection
   [junit4]   2> 49206 WARN  (OverseerThreadFactory-91-thread-1-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (nodeset_collection) without cores.
   [junit4]   2> 49322 INFO  (qtp141464611-258) [n:127.0.0.1:51561_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 49323 INFO  (qtp141464611-258) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=nodeset_collection&action=CREATE&numShards=2&createNodeSet=http://127.0.0.1:51561/solr,http://127.0.0.1:49074/solr&wt=javabin&version=2} status=0 QTime=121
   [junit4]   2> 49325 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.testCreateNodeSet-seed#[C8F88680C76DA882]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testCreateNodeSet
   [junit4]   2> 49481 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[C8F88680C76DA882]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTooManyReplicas
   [junit4]   2> 49482 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[C8F88680C76DA882]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 49488 INFO  (qtp141464611-257) [n:127.0.0.1:51561_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=nodeset_collection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 49501 INFO  (OverseerCollectionConfigSetProcessor-97039619797942280-127.0.0.1:43051_solr-n_0000000000) [n:127.0.0.1:43051_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 49502 INFO  (OverseerThreadFactory-91-thread-2-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 50210 INFO  (qtp141464611-257) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=nodeset_collection&action=DELETE&wt=javabin&version=2} status=0 QTime=722
   [junit4]   2> 50216 INFO  (qtp141464611-260) [n:127.0.0.1:51561_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 50218 INFO  (OverseerCollectionConfigSetProcessor-97039619797942280-127.0.0.1:43051_solr-n_0000000000) [n:127.0.0.1:43051_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 50219 INFO  (OverseerThreadFactory-91-thread-3-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.CreateCollectionCmd Create collection collection
   [junit4]   2> 50219 WARN  (OverseerThreadFactory-91-thread-3-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.CreateCollectionCmd Specified replicationFactor of 10 on collection collection is higher than or equal to the number of Solr instances currently live or live and part of your createNodeSet(4). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 50220 ERROR (OverseerThreadFactory-91-thread-3-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.OverseerCollectionMessageHandler Collection: collection operation: create failed:org.apache.solr.common.SolrException: Cannot create collection collection. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 4. This allows a maximum of 4 to be created. Value of numShards is 2 and value of replicationFactor is 10. This requires 20 shards to be created (higher than the allowed number)
   [junit4]   2> 	at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:148)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 50232 INFO  (qtp141464611-260) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2} status=400 QTime=16
   [junit4]   2> 50233 INFO  (OverseerCollectionConfigSetProcessor-97039619797942280-127.0.0.1:43051_solr-n_0000000000) [n:127.0.0.1:43051_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 50236 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[C8F88680C76DA882]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTooManyReplicas
   [junit4]   2> 50298 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI-seed#[C8F88680C76DA882]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCollectionsAPI
   [junit4]   2> 50299 INFO  (TEST-HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI-seed#[C8F88680C76DA882]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 50302 INFO  (qtp141464611-261) [n:127.0.0.1:51561_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=52f34ec4-18c8-4f22-b1e1-646421180edf&replicationFactor=4&collection.configName=conf2&maxShardsPerNode=5&name=awhollynewcollection_0&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 50308 INFO  (qtp141464611-261) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=52f34ec4-18c8-4f22-b1e1-646421180edf&replicationFactor=4&collection.configName=conf2&maxShardsPerNode=5&name=awhollynewcollection_0&action=CREATE&numShards=4&wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 50310 INFO  (qtp141464611-259) [n:127.0.0.1:51561_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=d7f1105b-d9d9-48f6-8e7d-a867c76a06a6&replicationFactor=4&collection.configName=conf2&maxShardsPerNode=6&name=awhollynewcollection_1&action=CREATE&numShards=5&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 50312 INFO  (qtp141464611-259) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=d7f1105b-d9d9-48f6-8e7d-a867c76a06a6&replicationFactor=4&collection.configName=conf2&maxShardsPerNode=6&name=awhollynewcollection_1&action=CREATE&numShards=5&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 50314 INFO  (qtp141464611-254) [n:127.0.0.1:51561_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=c5205c6e-fcb3-4155-a13d-992971d482f8&replicationFactor=2&collection.configName=conf2&maxShardsPerNode=3&name=awhollynewcollection_2&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 50316 INFO  (qtp141464611-254) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=c5205c6e-fcb3-4155-a13d-992971d482f8&replicationFactor=2&collection.configName=conf2&maxShardsPerNode=3&name=awhollynewcollection_2&action=CREATE&numShards=4&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 50318 INFO  (qtp141464611-258) [n:127.0.0.1:51561_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=db9cdea9-d7d8-4242-93b9-2144ed5340c8&replicationFactor=3&collection.configName=conf2&maxShardsPerNode=2&name=awhollynewcollection_3&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 50321 INFO  (qtp141464611-258) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=db9cdea9-d7d8-4242-93b9-2144ed5340c8&replicationFactor=3&collection.configName=conf2&maxShardsPerNode=2&name=awhollynewcollection_3&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 50323 INFO  (OverseerThreadFactory-91-thread-3-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.CreateCollectionCmd Create collection awhollynewcollection_2
   [junit4]   2> 50323 INFO  (qtp141464611-257) [n:127.0.0.1:51561_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=20cc0894-230e-4439-91d8-2713d7f7766e&replicationFactor=1&collection.configName=conf2&maxShardsPerNode=1&name=awhollynewcollection_4&action=CREATE&numShards=3&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 50327 INFO  (OverseerThreadFactory-91-thread-4-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.CreateCollectionCmd Create collection awhollynewcollection_0
   [junit4]   2> 50329 INFO  (qtp141464611-257) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=20cc0894-230e-4439-91d8-2713d7f7766e&replicationFactor=1&collection.configName=conf2&maxShardsPerNode=1&name=awhollynewcollection_4&action=CREATE&numShards=3&wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 50334 INFO  (OverseerThreadFactory-91-thread-2-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.CreateCollectionCmd Create collection awhollynewcollection_3
   [junit4]   2> 50348 INFO  (OverseerThreadFactory-91-thread-1-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.CreateCollectionCmd Create collection awhollynewcollection_4
   [junit4]   2> 50348 INFO  (OverseerThreadFactory-91-thread-5-processing-n:127.0.0.1:43051_solr) [n:127.0.0.1:43051_solr    ] o.a.s.c.CreateCollectionCmd Create collection awhollynewcollection_1
   [junit4]   2> 50378 INFO  (zkCallback-65-thread-5) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/awhollynewcollection_0/state.json] for collection [awhollynewcollection_0] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 50456 INFO  (qtp141464611-255) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=c5205c6e-fcb3-4155-a13d-992971d482f819083351701974062&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_2_shard4_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_2&shard=shard4&wt=javabin&version=2} status=0 QTime=12
   [junit4]   2> 50457 INFO  (parallelCoreAdminExecutor-96-thread-1-processing-n:127.0.0.1:51561_solr c5205c6e-fcb3-4155-a13d-992971d482f819083351701974062 CREATE) [n:127.0.0.1:51561_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50495 INFO  (qtp370359930-268) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=c5205c6e-fcb3-4155-a13d-992971d482f819083351705098189&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_2_shard2_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_2&shard=shard2&wt=javabin&version=2} status=0 QTime=30
   [junit4]   2> 50496 INFO  (parallelCoreAdminExecutor-92-thread-1-processing-n:127.0.0.1:44541_solr c5205c6e-fcb3-4155-a13d-992971d482f819083351705098189 CREATE) [n:127.0.0.1:44541_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50507 INFO  (qtp370359930-268) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=db9cdea9-d7d8-4242-93b9-2144ed5340c819083351715095648&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_3_shard1_replica1&action=CREATE&numShards=2&collection=awhollynewcollection_3&shard=shard1&wt=javabin&version=2} status=0 QTime=11
   [junit4]   2> 50508 INFO  (parallelCoreAdminExecutor-92-thread-2-processing-n:127.0.0.1:44541_solr db9cdea9-d7d8-4242-93b9-2144ed5340c819083351715095648 CREATE) [n:127.0.0.1:44541_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50510 INFO  (qtp370359930-268) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351744681380&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard1_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard1&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 50511 INFO  (qtp370359930-268) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351738052334&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard2_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard2&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 50511 INFO  (parallelCoreAdminExecutor-92-thread-3-processing-n:127.0.0.1:44541_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351744681380 CREATE) [n:127.0.0.1:44541_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50512 INFO  (qtp294931899-243) [n:127.0.0.1:49074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=c5205c6e-fcb3-4155-a13d-992971d482f819083351709759843&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_2_shard1_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_2&shard=shard1&wt=javabin&version=2} status=0 QTime=46
   [junit4]   2> 50512 INFO  (parallelCoreAdminExecutor-92-thread-4-processing-n:127.0.0.1:44541_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351738052334 CREATE) [n:127.0.0.1:44541_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50513 INFO  (parallelCoreAdminExecutor-89-thread-1-processing-n:127.0.0.1:49074_solr c5205c6e-fcb3-4155-a13d-992971d482f819083351709759843 CREATE) [n:127.0.0.1:49074_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50519 INFO  (qtp370359930-268) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351743446797&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard4_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard4&wt=javabin&version=2} status=0 QTime=7
   [junit4]   2> 50523 INFO  (qtp370359930-268) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=db9cdea9-d7d8-4242-93b9-2144ed5340c819083351717624907&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_3_shard2_replica2&action=CREATE&numShards=2&collection=awhollynewcollection_3&shard=shard2&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 50532 INFO  (parallelCoreAdminExecutor-92-thread-6-processing-n:127.0.0.1:44541_solr db9cdea9-d7d8-4242-93b9-2144ed5340c819083351717624907 CREATE) [n:127.0.0.1:44541_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50532 INFO  (parallelCoreAdminExecutor-92-thread-5-processing-n:127.0.0.1:44541_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351743446797 CREATE) [n:127.0.0.1:44541_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50532 INFO  (qtp1720077198-233) [n:127.0.0.1:43051_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=c5205c6e-fcb3-4155-a13d-992971d482f819083351702567144&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_2_shard3_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_2&shard=shard3&wt=javabin&version=2} status=0 QTime=7
   [junit4]   2> 50532 INFO  (parallelCoreAdminExecutor-94-thread-1-processing-n:127.0.0.1:43051_solr c5205c6e-fcb3-4155-a13d-992971d482f819083351702567144 CREATE) [n:127.0.0.1:43051_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50547 INFO  (qtp294931899-243) [n:127.0.0.1:49074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351725923514&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard4_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard4&wt=javabin&version=2} status=0 QTime=35
   [junit4]   2> 50567 INFO  (parallelCoreAdminExecutor-89-thread-2-processing-n:127.0.0.1:49074_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351725923514 CREATE) [n:127.0.0.1:49074_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50570 INFO  (qtp370359930-268) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=20cc0894-230e-4439-91d8-2713d7f7766e19083351833279769&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_4_shard2_replica1&action=CREATE&numShards=3&collection=awhollynewcollection_4&shard=shard2&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 50570 INFO  (parallelCoreAdminExecutor-92-thread-7-processing-n:127.0.0.1:44541_solr 20cc0894-230e-4439-91d8-2713d7f7766e19083351833279769 CREATE) [n:127.0.0.1:44541_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50571 INFO  (qtp141464611-255) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351742356386&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard2_replica4&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard2&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 50578 INFO  (parallelCoreAdminExecutor-96-thread-5-processing-n:127.0.0.1:51561_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351742356386 CREATE) [n:127.0.0.1:51561_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50587 INFO  (qtp141464611-254) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=20cc0894-230e-4439-91d8-2713d7f7766e19083351740618086&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_4_shard3_replica1&action=CREATE&numShards=3&collection=awhollynewcollection_4&shard=shard3&wt=javabin&version=2} status=0 QTime=8
   [junit4]   2> 50589 INFO  (parallelCoreAdminExecutor-96-thread-8-processing-n:127.0.0.1:51561_solr 20cc0894-230e-4439-91d8-2713d7f7766e19083351740618086 CREATE) [n:127.0.0.1:51561_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50591 INFO  (qtp141464611-254) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351740589535&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard4_replica4&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard4&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 50599 INFO  (qtp141464611-257) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=c5205c6e-fcb3-4155-a13d-992971d482f819083351709396512&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_2_shard2_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_2&shard=shard2&wt=javabin&version=2} status=0 QTime=30
   [junit4]   2> 50600 INFO  (qtp141464611-260) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=db9cdea9-d7d8-4242-93b9-2144ed5340c819083351711221410&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_3_shard1_replica2&action=CREATE&numShards=2&collection=awhollynewcollection_3&shard=shard1&wt=javabin&version=2} status=0 QTime=131
   [junit4]   2> 50600 INFO  (parallelCoreAdminExecutor-96-thread-3-processing-n:127.0.0.1:51561_solr db9cdea9-d7d8-4242-93b9-2144ed5340c819083351711221410 CREATE) [n:127.0.0.1:51561_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50600 INFO  (parallelCoreAdminExecutor-96-thread-4-processing-n:127.0.0.1:51561_solr c5205c6e-fcb3-4155-a13d-992971d482f819083351709396512 CREATE) [n:127.0.0.1:51561_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50600 INFO  (qtp141464611-261) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351725840300&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard3_replica4&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard3&wt=javabin&version=2} status=0 QTime=31
   [junit4]   2> 50607 INFO  (qtp370359930-273) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351717510759&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard3_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard3&wt=javabin&version=2} status=0 QTime=41
   [junit4]   2> 50608 INFO  (parallelCoreAdminExecutor-92-thread-9-processing-n:127.0.0.1:44541_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351717510759 CREATE) [n:127.0.0.1:44541_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50608 INFO  (qtp294931899-242) [n:127.0.0.1:49074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=c5205c6e-fcb3-4155-a13d-992971d482f819083351706685966&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_2_shard3_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_2&shard=shard3&wt=javabin&version=2} status=0 QTime=75
   [junit4]   2> 50609 INFO  (parallelCoreAdminExecutor-89-thread-3-processing-n:127.0.0.1:49074_solr c5205c6e-fcb3-4155-a13d-992971d482f819083351706685966 CREATE) [n:127.0.0.1:49074_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50610 INFO  (qtp1720077198-236) [n:127.0.0.1:43051_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351739237064&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard4_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard4&wt=javabin&version=2} status=0 QTime=40
   [junit4]   2> 50610 INFO  (parallelCoreAdminExecutor-94-thread-2-processing-n:127.0.0.1:43051_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351739237064 CREATE) [n:127.0.0.1:43051_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50610 INFO  (qtp370359930-232) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=c5205c6e-fcb3-4155-a13d-992971d482f819083351702856560&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_2_shard4_replica1&action=CREATE&numShards=4&collection=awhollynewcollection_2&shard=shard4&wt=javabin&version=2} status=0 QTime=132
   [junit4]   2> 50611 INFO  (parallelCoreAdminExecutor-92-thread-8-processing-n:127.0.0.1:44541_solr c5205c6e-fcb3-4155-a13d-992971d482f819083351702856560 CREATE) [n:127.0.0.1:44541_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50611 INFO  (parallelCoreAdminExecutor-96-thread-9-processing-n:127.0.0.1:51561_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351740589535 CREATE) [n:127.0.0.1:51561_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50699 INFO  (qtp141464611-255) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351741747942&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard1_replica4&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard1&wt=javabin&version=2} status=0 QTime=128
   [junit4]   2> 50705 INFO  (qtp141464611-259) [n:127.0.0.1:51561_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=db9cdea9-d7d8-4242-93b9-2144ed5340c819083351710595954&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_3_shard2_replica3&action=CREATE&numShards=2&collection=awhollynewcollection_3&shard=shard2&wt=javabin&version=2} status=0 QTime=135
   [junit4]   2> 50706 INFO  (parallelCoreAdminExecutor-96-thread-6-processing-n:127.0.0.1:51561_solr db9cdea9-d7d8-4242-93b9-2144ed5340c819083351710595954 CREATE) [n:127.0.0.1:51561_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50708 INFO  (parallelCoreAdminExecutor-96-thread-7-processing-n:127.0.0.1:51561_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351741747942 CREATE) [n:127.0.0.1:51561_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50720 INFO  (qtp294931899-245) [n:127.0.0.1:49074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=20cc0894-230e-4439-91d8-2713d7f7766e19083351833133776&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_4_shard1_replica1&action=CREATE&numShards=3&collection=awhollynewcollection_4&shard=shard1&wt=javabin&version=2} status=0 QTime=45
   [junit4]   2> 50721 INFO  (parallelCoreAdminExecutor-89-thread-6-processing-n:127.0.0.1:49074_solr 20cc0894-230e-4439-91d8-2713d7f7766e19083351833133776 CREATE) [n:127.0.0.1:49074_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50723 INFO  (qtp294931899-249) [n:127.0.0.1:49074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351741207575&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard1_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard1&wt=javabin&version=2} status=0 QTime=190
   [junit4]   2> 50724 INFO  (parallelCoreAdminExecutor-89-thread-5-processing-n:127.0.0.1:49074_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351741207575 CREATE) [n:127.0.0.1:49074_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50725 INFO  (qtp294931899-248) [n:127.0.0.1:49074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=db9cdea9-d7d8-4242-93b9-2144ed5340c819083351716754631&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_3_shard2_replica1&action=CREATE&numShards=2&collection=awhollynewcollection_3&shard=shard2&wt=javabin&version=2} status=0 QTime=192
   [junit4]   2> 50726 INFO  (parallelCoreAdminExecutor-89-thread-4-processing-n:127.0.0.1:49074_solr db9cdea9-d7d8-4242-93b9-2144ed5340c819083351716754631 CREATE) [n:127.0.0.1:49074_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50726 INFO  (qtp1720077198-237) [n:127.0.0.1:43051_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=db9cdea9-d7d8-4242-93b9-2144ed5340c819083351709107907&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_3_shard1_replica3&action=CREATE&numShards=2&collection=awhollynewcollection_3&shard=shard1&wt=javabin&version=2} status=0 QTime=201
   [junit4]   2> 50726 INFO  (qtp1720077198-231) [n:127.0.0.1:43051_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=c5205c6e-fcb3-4155-a13d-992971d482f819083351708182222&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_2_shard1_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_2&shard=shard1&wt=javabin&version=2} status=0 QTime=156
   [junit4]   2> 50741 INFO  (qtp370359930-274) [n:127.0.0.1:44541_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 20cc0894-230e-4439-91d8-2713d7f7766e19083351833279769
   [junit4]   2> 50750 INFO  (qtp370359930-274) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=20cc0894-230e-4439-91d8-2713d7f7766e19083351833279769&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=9
   [junit4]   2> 50747 INFO  (qtp294931899-246) [n:127.0.0.1:49074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351732230461&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard3_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard3&wt=javabin&version=2} status=0 QTime=8
   [junit4]   2> 50749 INFO  (parallelCoreAdminExecutor-89-thread-8-processing-n:127.0.0.1:49074_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351745286733 CREATE) [n:127.0.0.1:49074_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50749 INFO  (parallelCoreAdminExecutor-89-thread-7-processing-n:127.0.0.1:49074_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351732230461 CREATE) [n:127.0.0.1:49074_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50749 INFO  (parallelCoreAdminExecutor-94-thread-4-processing-n:127.0.0.1:43051_solr db9cdea9-d7d8-4242-93b9-2144ed5340c819083351709107907 CREATE) [n:127.0.0.1:43051_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50748 INFO  (parallelCoreAdminExecutor-94-thread-3-processing-n:127.0.0.1:43051_solr c5205c6e-fcb3-4155-a13d-992971d482f819083351708182222 CREATE) [n:127.0.0.1:43051_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50748 INFO  (parallelCoreAdminExecutor-96-thread-2-processing-n:127.0.0.1:51561_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351725840300 CREATE) [n:127.0.0.1:51561_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50748 INFO  (qtp294931899-247) [n:127.0.0.1:49074_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351745286733&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard2_replica2&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard2&wt=javabin&version=2} status=0 QTime=9
   [junit4]   2> 50776 INFO  (qtp1720077198-238) [n:127.0.0.1:43051_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351736815554&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard3_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard3&wt=javabin&version=2} status=0 QTime=27
   [junit4]   2> 50777 INFO  (parallelCoreAdminExecutor-94-thread-6-processing-n:127.0.0.1:43051_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351736815554 CREATE) [n:127.0.0.1:43051_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50787 INFO  (qtp1720077198-235) [n:127.0.0.1:43051_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351744070805&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard2_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard2&wt=javabin&version=2} status=0 QTime=216
   [junit4]   2> 50787 INFO  (parallelCoreAdminExecutor-94-thread-5-processing-n:127.0.0.1:43051_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351744070805 CREATE) [n:127.0.0.1:43051_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50788 INFO  (qtp1720077198-239) [n:127.0.0.1:43051_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=52f34ec4-18c8-4f22-b1e1-646421180edf19083351742905553&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_0_shard1_replica3&action=CREATE&numShards=4&collection=awhollynewcollection_0&shard=shard1&wt=javabin&version=2} status=0 QTime=27
   [junit4]   2> 50788 INFO  (parallelCoreAdminExecutor-94-thread-7-processing-n:127.0.0.1:43051_solr 52f34ec4-18c8-4f22-b1e1-646421180edf19083351742905553 CREATE) [n:127.0.0.1:43051_solr    ] o.a.s.u.TestInjection Inject random core creation delay of 6s
   [junit4]   2> 50788 INFO  (qtp370359930-271) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=d7f1105b-d9d9-48f6-8e7d-a867c76a06a619083351884119046&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_1_shard5_replica2&action=CREATE&numShards=5&collection=awhollynewcollection_1&shard=shard5&wt=javabin&version=2} status=0 QTime=38
   [junit4]   2> 50791 INFO  (qtp1720077198-236) [n:127.0.0.1:43051_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : db9cdea9-d7d8-4242-93b9-2144ed5340c819083351709107907
   [junit4]   2> 50791 INFO  (qtp1720077198-233) [n:127.0.0.1:43051_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : c5205c6e-fcb3-4155-a13d-992971d482f819083351708182222
   [junit4]   2> 50791 INFO  (qtp1720077198-236) [n:127.0.0.1:43051_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=db9cdea9-d7d8-4242-93b9-2144ed5340c819083351709107907&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 50791 INFO  (qtp1720077198-233) [n:127.0.0.1:43051_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=c5205c6e-fcb3-4155-a13d-992971d482f819083351708182222&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 50791 INFO  (qtp1720077198-236) [n:127.0.0.1:43051_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 52f34ec4-18c8-4f22-b1e1-646421180edf19083351744070805
   [junit4]   2> 50791 INFO  (qtp1720077198-236) [n:127.0.0.1:43051_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=52f34ec4-18c8-4f22-b1e1-646421180edf19083351744070805&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 50802 INFO  (qtp370359930-270) [n:127.0.0.1:44541_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=d7f1105b-d9d9-48f6-8e7d-a867c76a06a619083352059056912&qt=/admin/cores&collection.configName=conf2&newCollection=true&name=awhollynewcollection_1_shard2_replica2&action=CREATE&numShards=5&collection=awhollynewcollection_1&shard=shard2&wt=javabin&version=2} status=0 QTime=5
   [junit4]   2> 50803 INFO  (parallelCoreAdminExecutor-92-thread-11

[...truncated too long message...]

yScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 272086 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 272188 WARN  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to localhost/127.0.0.1:47085) [    ] o.a.h.h.s.d.DataNode BPOfferService for Block pool BP-835711701-10.41.0.5-1480707092891 (Datanode Uuid 68dab75c-e896-408f-bb83-94caf7ae844a) service to localhost/127.0.0.1:47085 interrupted
   [junit4]   2> 272188 WARN  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to localhost/127.0.0.1:47085) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-835711701-10.41.0.5-1480707092891 (Datanode Uuid 68dab75c-e896-408f-bb83-94caf7ae844a) service to localhost/127.0.0.1:47085
   [junit4]   2> 272222 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 272326 ERROR (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
   [junit4]   2> java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:401)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:194)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
   [junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
   [junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2> 	at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
   [junit4]   2> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
   [junit4]   2> 	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
   [junit4]   2> 	at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
   [junit4]   2> 	... 54 more
   [junit4]   2> 272332 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:48373 48373
   [junit4]   2> 272332 INFO  (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C8F88680C76DA882]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C8F88680C76DA882-001
   [junit4]   2> Dec 02, 2016 7:35:22 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1230, maxMBSortInHeap=6.166455681148836, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=ar-SD, timezone=GB
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=5,free=246385952,total=373817344
   [junit4]   2> NOTE: All tests run in this JVM: [TestJavabinTupleStreamParser, TestDefaultSearchFieldResource, RecoveryZkTest, ConnectionManagerTest, MinimalSchemaTest, HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [44/658 (1!)] on J2 in 242.88s, 20 tests, 1 failure, 1 error <<< FAILURES!

[...truncated 62695 lines...]



Mime
View raw message