lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 811 - Still Failing
Date Sat, 03 Oct 2015 17:33:17 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/811/

4 tests failed.
FAILED:  org.apache.solr.cloud.CdcrReplicationHandlerTest.doTest

Error Message:
Captured an uncaught exception in thread: Thread[id=26980, name=RecoveryThread-source_collection_shard1_replica1, state=RUNNABLE, group=TGRP-CdcrReplicationHandlerTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=26980, name=RecoveryThread-source_collection_shard1_replica1, state=RUNNABLE, group=TGRP-CdcrReplicationHandlerTest]
Caused by: org.apache.solr.common.cloud.ZooKeeperException: 
	at __randomizedtesting.SeedInfo.seed([AD694E9F3B68A24F]:0)
	at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:232)
Caused by: org.apache.solr.common.SolrException: java.io.FileNotFoundException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CdcrReplicationHandlerTest_AD694E9F3B68A24F-001/jetty-001/cores/source_collection_shard1_replica1/data/tlog/tlog.0000000000000000008.1514030901672017920 (No such file or directory)
	at org.apache.solr.update.CdcrTransactionLog.reopenOutputStream(CdcrTransactionLog.java:244)
	at org.apache.solr.update.CdcrTransactionLog.incref(CdcrTransactionLog.java:173)
	at org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1079)
	at org.apache.solr.update.UpdateLog.seedBucketsWithHighestVersion(UpdateLog.java:1579)
	at org.apache.solr.update.UpdateLog.seedBucketsWithHighestVersion(UpdateLog.java:1610)
	at org.apache.solr.core.SolrCore.seedVersionBuckets(SolrCore.java:877)
	at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:534)
	at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:225)
Caused by: java.io.FileNotFoundException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.CdcrReplicationHandlerTest_AD694E9F3B68A24F-001/jetty-001/cores/source_collection_shard1_replica1/data/tlog/tlog.0000000000000000008.1514030901672017920 (No such file or directory)
	at java.io.RandomAccessFile.open0(Native Method)
	at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
	at org.apache.solr.update.CdcrTransactionLog.reopenOutputStream(CdcrTransactionLog.java:236)
	... 7 more


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:35817/_n: Could not load collection from ZK:nodes_used_collection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35817/_n: Could not load collection from ZK:nodes_used_collection
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:302)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:419)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest.test

Error Message:
commitWithin did not work on node: http://127.0.0.1:37834/j_rq/d/collection1 expected:<68> but was:<67>

Stack Trace:
java.lang.AssertionError: commitWithin did not work on node: http://127.0.0.1:37834/j_rq/d/collection1 expected:<68> but was:<67>
	at __randomizedtesting.SeedInfo.seed([AD694E9F3B68A24F:253D71459594CFB7]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:333)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsSyncSliceTest.test

Error Message:
timeout waiting to see all nodes active

Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
	at __randomizedtesting.SeedInfo.seed([AD694E9F3B68A24F:253D71459594CFB7]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SyncSliceTest.waitTillAllNodesActive(SyncSliceTest.java:239)
	at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10384 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/init-core-data-001
   [junit4]   2> 1654663 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /j_rq/d
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1654755 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1654771 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1654772 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1654796 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_44773_hdfs____grh9j1/webapp
   [junit4]   2> 1655036 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1655411 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44773
   [junit4]   2> 1655593 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1655594 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1655607 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_57647_datanode____.cfi81u/webapp
   [junit4]   2> 1655715 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1655965 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57647
   [junit4]   2> 1656108 WARN  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1656109 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1656132 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_57354_datanode____.a00b8p/webapp
   [junit4]   2> 1656251 INFO  (IPC Server handler 2 on 45938) [    ] BlockStateChange BLOCK* processReport: from storage DS-82df38fb-bac3-449e-b355-883387f43a61 node DatanodeRegistration(127.0.0.1, datanodeUuid=70871bf9-9106-48da-b510-80ef2080043f, infoPort=57647, ipcPort=35448, storageInfo=lv=-56;cid=testClusterID;nsid=1426229331;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 1656251 INFO  (IPC Server handler 2 on 45938) [    ] BlockStateChange BLOCK* processReport: from storage DS-31c0ab71-9523-4dfc-8e54-aecd4939bc70 node DatanodeRegistration(127.0.0.1, datanodeUuid=70871bf9-9106-48da-b510-80ef2080043f, infoPort=57647, ipcPort=35448, storageInfo=lv=-56;cid=testClusterID;nsid=1426229331;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1656298 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1656533 INFO  (SUITE-HdfsBasicDistributedZkTest-seed#[AD694E9F3B68A24F]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57354
   [junit4]   2> 1656671 INFO  (IPC Server handler 5 on 45938) [    ] BlockStateChange BLOCK* processReport: from storage DS-04e47c6a-45e2-49a7-84d2-23ce1222e8c8 node DatanodeRegistration(127.0.0.1, datanodeUuid=2217bf0b-9169-4940-80d6-43e8ba020601, infoPort=57354, ipcPort=60939, storageInfo=lv=-56;cid=testClusterID;nsid=1426229331;c=0), blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 1656672 INFO  (IPC Server handler 5 on 45938) [    ] BlockStateChange BLOCK* processReport: from storage DS-d3d2b09a-d174-4a60-86c3-b2e36d612e12 node DatanodeRegistration(127.0.0.1, datanodeUuid=2217bf0b-9169-4940-80d6-43e8ba020601, infoPort=57354, ipcPort=60939, storageInfo=lv=-56;cid=testClusterID;nsid=1426229331;c=0), blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1656779 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1656780 INFO  (Thread-12723) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1656780 INFO  (Thread-12723) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1656880 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.ZkTestServer start zk server on port:57727
   [junit4]   2> 1656880 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1656881 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1656884 INFO  (zkCallback-1465-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@539eb9db name:ZooKeeperConnection Watcher:127.0.0.1:57727 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1656884 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1656884 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1656884 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1656888 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1656895 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1656896 INFO  (zkCallback-1466-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@479b56e6 name:ZooKeeperConnection Watcher:127.0.0.1:57727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1656896 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1656896 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1656896 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1656899 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1656900 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1656900 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1656901 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1656902 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1656903 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1656903 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1656905 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1656905 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1656906 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1656906 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1656907 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1656907 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1656908 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1656908 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1656909 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1656910 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1656911 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1656911 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1656912 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1656912 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1656913 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1656913 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1656914 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1656914 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1657258 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/cores/collection1
   [junit4]   2> 1657260 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1657261 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@227210a5{/j_rq/d,null,AVAILABLE}
   [junit4]   2> 1657262 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@41d52aff{HTTP/1.1}{127.0.0.1:52424}
   [junit4]   2> 1657262 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.e.j.s.Server Started @1660649ms
   [junit4]   2> 1657262 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:45938/hdfs__localhost_45938__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001_tempDir-002_control_data, hostContext=/j_rq/d, hostPort=52424, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/cores}
   [junit4]   2> 1657262 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1657262 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/'
   [junit4]   2> 1657277 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1657278 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1657278 INFO  (zkCallback-1467-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@23e0739a name:ZooKeeperConnection Watcher:127.0.0.1:57727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1657278 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1657279 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1657279 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1657279 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/solr.xml
   [junit4]   2> 1657284 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/cores
   [junit4]   2> 1657284 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.CoreContainer New CoreContainer 1092537763
   [junit4]   2> 1657284 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/]
   [junit4]   2> 1657284 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.CoreContainer loading shared library: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/lib
   [junit4]   2> 1657284 WARN  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/lib).
   [junit4]   2> 1657290 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1657291 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1657292 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1657292 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1657292 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:57727/solr
   [junit4]   2> 1657292 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1657292 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1657295 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1657299 INFO  (zkCallback-1469-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@65798ef5 name:ZooKeeperConnection Watcher:127.0.0.1:57727 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1657299 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1657299 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1657307 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1657310 INFO  (zkCallback-1470-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@50f5b3bf name:ZooKeeperConnection Watcher:127.0.0.1:57727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1657310 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1657311 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1657312 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1657315 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1657316 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1657318 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1657320 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1657321 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1657321 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1657322 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1657323 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52424_j_rq%2Fd
   [junit4]   2> 1657323 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52424_j_rq%2Fd
   [junit4]   2> 1657324 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1657325 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1657326 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1657326 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000
   [junit4]   2> 1657327 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52424_j_rq%2Fd
   [junit4]   2> 1657327 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1657328 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.Overseer Overseer (id=94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) starting
   [junit4]   2> 1657329 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1657333 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1657339 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1657339 INFO  (OverseerCollectionConfigSetProcessor-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1657340 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1657343 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1657361 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1657361 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1657362 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.CoresLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/cores
   [junit4]   2> 1657362 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1657363 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.CoresLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/cores/collection1/
   [junit4]   2> 1657363 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1657364 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1657364 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1657365 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 1657365 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52424/j_rq/d",
   [junit4]   2>   "node_name":"127.0.0.1:52424_j_rq%2Fd",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1657366 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1657366 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52424/j_rq/d",
   [junit4]   2>   "node_name":"127.0.0.1:52424_j_rq%2Fd",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1657366 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1657366 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1657367 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1657368 INFO  (zkCallback-1470-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1657368 INFO  (zkCallback-1470-thread-2-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1658366 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1658367 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1658367 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1658367 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1658368 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1658368 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/cores/collection1/'
   [junit4]   2> 1658388 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1658393 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1658396 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1658404 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1658405 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1658409 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1658576 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1658581 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1658582 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1658590 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1658592 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1658593 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1658594 WARN  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1658594 WARN  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1658594 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1658595 WARN  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1658595 WARN  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1658596 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 1658596 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1658596 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 1658596 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:45938/solr_hdfs_home
   [junit4]   2> 1658596 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1658596 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 1658596 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c6692fb
   [junit4]   2> 1658597 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45938/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1658614 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:45938/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1658616 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:45938/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 1658624 WARN  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:45938/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1658625 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45938/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1658632 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 1658632 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1658632 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 1658638 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1658638 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:45938/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1658638 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=19.4404296875, floorSegmentMB=1.833984375, forceMergeDeletesPctAllowed=8.282826214347109, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1658690 INFO  (IPC Server handler 4 on 45938) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50748 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-82df38fb-bac3-449e-b355-883387f43a61:NORMAL:127.0.0.1:35919|RBW], ReplicaUnderConstruction[[DISK]DS-d3d2b09a-d174-4a60-86c3-b2e36d612e12:NORMAL:127.0.0.1:50748|FINALIZED]]} size 0
   [junit4]   2> 1658692 INFO  (IPC Server handler 7 on 45938) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35919 is added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[[DISK]DS-82df38fb-bac3-449e-b355-883387f43a61:NORMAL:127.0.0.1:35919|RBW], ReplicaUnderConstruction[[DISK]DS-d3d2b09a-d174-4a60-86c3-b2e36d612e12:NORMAL:127.0.0.1:50748|FINALIZED]]} size 0
   [junit4]   2> 1658710 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b77f7b60 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74696672); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1658711 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1658718 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1658718 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1658718 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1658718 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1658718 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1658718 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1658719 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1658719 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1658719 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1658721 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1658721 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1658722 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1658723 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1658725 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 1658726 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1658726 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:45938/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 1658736 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1658736 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 1658766 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1658766 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1658767 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=24, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=15.283203125, floorSegmentMB=0.552734375, forceMergeDeletesPctAllowed=8.630558536476492, segmentsPerTier=6.0, maxCFSSegmentSizeMB=0.4365234375, noCFSRatio=0.4679978822246281
   [junit4]   2> 1658803 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b77f7b60 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74696672); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1658803 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1658803 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@dbb0a14[collection1] main
   [junit4]   2> 1658803 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from:/collections/control_collection
   [junit4]   2> 1658807 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1658807 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1658807 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1658808 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1658808 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1658808 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1658808 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1658808 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1658808 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1658809 INFO  (searcherExecutor-6632-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@dbb0a14[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1658809 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1658809 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version buckets from index
   [junit4]   2> 1658810 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1658810 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1514029063909408768
   [junit4]   2> 1658810 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 1514029063909408768
   [junit4]   2> 1658810 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1658810 INFO  (coreLoadExecutor-6631-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1658835 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:52424/j_rq/d collection:control_collection shard:shard1
   [junit4]   2> 1658836 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1658842 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/94626816361365508-core_node1-n_0000000000
   [junit4]   2> 1658842 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1658844 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1658849 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1658849 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1658849 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52424/j_rq/d/collection1/
   [junit4]   2> 1658849 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1658849 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:52424/j_rq/d/collection1/ has no replicas
   [junit4]   2> 1658849 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52424/j_rq/d/collection1/ shard1
   [junit4]   2> 1658849 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1658851 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1658853 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:52424/j_rq/d",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1658868 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1658892 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1658893 INFO  (zkCallback-1472-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1836cf2f name:ZooKeeperConnection Watcher:127.0.0.1:57727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1658893 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1658893 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1658893 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1658908 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1658908 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1658908 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1658927 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1658940 INFO  (zkCallback-1473-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5fa0ee02 name:ZooKeeperConnection Watcher:127.0.0.1:57727/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1658940 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1658941 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1658947 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 1
   [junit4]   2> 1658947 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1658948 INFO  (zkCallback-1472-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1658948 INFO  (zkCallback-1470-thread-2-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1658948 INFO  (zkCallback-1470-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1658954 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:52424/j_rq/d/collection1/ and leader is http://127.0.0.1:52424/j_rq/d/collection1/
   [junit4]   2> 1658954 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52424/j_rq/d
   [junit4]   2> 1658954 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1658955 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1658955 INFO  (coreZkRegister-6625-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52424_j_rq%2Fd c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1658956 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52424/j_rq/d",
   [junit4]   2>   "node_name":"127.0.0.1:52424_j_rq%2Fd",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1658957 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52424/j_rq/d",
   [junit4]   2>   "node_name":"127.0.0.1:52424_j_rq%2Fd",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1658957 INFO  (OverseerStateUpdate-94626816361365508-127.0.0.1:52424_j_rq%2Fd-n_0000000000) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 1659060 INFO  (zkCallback-1472-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1659060 INFO  (zkCallback-1470-thread-2-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1659060 INFO  (zkCallback-1470-thread-1-processing-n:127.0.0.1:52424_j_rq%2Fd) [n:127.0.0.1:52424_j_rq%2Fd    ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1659262 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/shard-1-001/cores/collection1
   [junit4]   2> 1659263 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/shard-1-001
   [junit4]   2> 1659264 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1659266 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@74af41b6{/j_rq/d,null,AVAILABLE}
   [junit4]   2> 1659266 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.e.j.s.ServerConnector Started ServerConnector@4ab42c0d{HTTP/1.1}{127.0.0.1:37834}
   [junit4]   2> 1659266 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.e.j.s.Server Started @1662654ms
   [junit4]   2> 1659266 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:45938/hdfs__localhost_45938__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/j_rq/d, hostPort=37834, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/shard-1-001/cores}
   [junit4]   2> 1659266 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1659269 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AD694E9F3B68A24F-001/shard-1-001/'
   [junit4]   2> 1659283 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1659284 INFO  (TEST-HdfsBasicDistributedZkTest.test-seed#[AD694E9F3B68A24F]) [    ] o.a.s.c.c.Conn

[...truncated too long message...]

rImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2> 	... 54 more
   [junit4]   2> 4309655 INFO  (SUITE-HdfsSyncSliceTest-seed#[AD694E9F3B68A24F]-worker) [n:127.0.0.1:58162_d_f c:collection1 s:shard1 r:core_node7 x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsSyncSliceTest_AD694E9F3B68A24F-001
   [junit4]   2> Oct 03, 2015 1:28:03 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {other_tl1=PostingsFormat(name=Direct), range_facet_l_dv=PostingsFormat(name=Asserting), multiDefault=Lucene50(blocksize=128), intDefault=PostingsFormat(name=Direct), a_i1=Lucene50(blocksize=128), range_facet_l=Lucene50(blocksize=128), rnd_b=PostingsFormat(name=Asserting), _version_=PostingsFormat(name=Direct), a_t=Lucene50(blocksize=128), id=PostingsFormat(name=Asserting), range_facet_i_dv=Lucene50(blocksize=128), text=Lucene50(blocksize=128), timestamp=Lucene50(blocksize=128)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting)}, sim=ClassicSimilarity, locale=, timezone=SystemV/CST6CDT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=4,threads=2,free=168571904,total=524288000
   [junit4]   2> NOTE: All tests run in this JVM: [DistributedFacetPivotLargeTest, TestCSVLoader, EnumFieldTest, PathHierarchyTokenizerFactoryTest, TestSortingResponseWriter, CachingDirectoryFactoryTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestEmbeddedSolrServerConstructors, TestSolrConfigHandlerCloud, SimpleCollectionCreateDeleteTest, TestChildDocTransformer, TestDistributedSearch, TestStressLucene, TestCloudManagedSchemaConcurrent, TestAtomicUpdateErrorCases, SynonymTokenizerTest, PeerSyncTest, RankQueryTest, RequiredFieldsTest, TestLMDirichletSimilarityFactory, ReplicationFactorTest, SyncSliceTest, TestIntervalFaceting, TestRandomFaceting, TestArbitraryIndexDir, DefaultValueUpdateProcessorTest, TestDocSet, AlternateDirectoryTest, TestSimpleQParserPlugin, CoreMergeIndexesAdminHandlerTest, TestDocumentBuilder, IndexSchemaTest, ExitableDirectoryReaderTest, DateFieldTest, LeaderElectionIntegrationTest, TestLeaderElectionZkExpiry, PrimUtilsTest, DistributedQueryElevationComponentTest, SparseHLLTest, HdfsBasicDistributedZk2Test, SpellingQueryConverterTest, ResourceLoaderTest, TestAnalyzedSuggestions, TestFunctionQuery, TestClusterStateMutator, TestJoin, TestSolrConfigHandlerConcurrent, DeleteLastCustomShardedReplicaTest, TestExceedMaxTermLength, TestMiniSolrCloudClusterSSL, TestExclusionRuleCollectionAccess, UUIDUpdateProcessorFallbackTest, FileUtilsTest, BigEndianAscendingWordSerializerTest, CurrencyFieldOpenExchangeTest, BaseCdcrDistributedZkTest, CollectionStateFormat2Test, ReturnFieldsTest, DistribCursorPagingTest, DistributedSuggestComponentTest, TestDynamicFieldResource, TestSolrQueryParserDefaultOperatorResource, TestBadConfig, TestReplicationHandler, ZkStateWriterTest, TestPhraseSuggestions, TestExactStatsCache, TestBlobHandler, StandardRequestHandlerTest, TestInfoStreamLogging, AliasIntegrationTest, DisMaxRequestHandlerTest, TestTrackingShardHandlerFactory, BasicDistributedZkTest, TestPseudoReturnFields, CdcrReplicationDistributedZkTest, RemoteQueryErrorTest, DistributedQueryComponentOptimizationTest, TestSchemaResource, TestRandomMergePolicy, BasicZkTest, LeaderInitiatedRecoveryOnCommitTest, TestInitParams, AnalysisAfterCoreReloadTest, QueryElevationComponentTest, TestStandardQParsers, DistributedVersionInfoTest, PingRequestHandlerTest, JSONWriterTest, SpellCheckCollatorTest, CoreAdminHandlerTest, PolyFieldTest, SolrCoreCheckLockOnStartupTest, FieldMutatingUpdateProcessorTest, QueryEqualityTest, TestRemoteStreaming, SolrInfoMBeanTest, DebugComponentTest, TestSurroundQueryParser, LukeRequestHandlerTest, PrimitiveFieldTypeTest, FileBasedSpellCheckerTest, TermsComponentTest, TestValueSourceCache, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, TestCollationField, JsonLoaderTest, TestComponentsName, HighlighterConfigTest, SOLR749Test, TestQuerySenderListener, UpdateParamsTest, TestSolrIndexConfig, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, TestMergePolicyConfig, TestBinaryField, TestElisionMultitermQuery, TestConfig, OutputWriterTest, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestXIncludeConfig, EchoParamsTest, TestPerFieldSimilarity, TestIBSimilarityFactory, TimeZoneUtilsTest, TestFastOutputStream, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, DateMathParserTest, ChaosMonkeyNothingIsSafeTest, PreAnalyzedFieldTest, ClusterStateTest, TestSolrJ, TestLRUCache, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, CircularListTest, DistributedMLTComponentTest, TestRTGBase, CursorPagingTest, DistributedIntervalFacetingTest, CdcrRequestHandlerTest, CdcrVersionReplicationTest, CollectionTooManyReplicasTest, DeleteInactiveReplicaTest, DeleteReplicaTest, DistributedQueueTest, HttpPartitionTest, TestCloudInspectUtil, TestConfigSetsAPI, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestMiniSolrCloudCluster, TestReplicaProperties, TestSolrCloudWithKerberosAlt, HdfsSyncSliceTest]
   [junit4] Completed [488/546] on J0 in 300.01s, 1 test, 1 failure <<< FAILURES!

[...truncated 194 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:782: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:719: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:516: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1432: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:987: There were test failures: 546 suites (8 ignored), 2145 tests, 2 errors, 2 failures, 29 ignored (17 assumptions) [seed: AD694E9F3B68A24F]

Total time: 214 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Mime
View raw message