lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_20) - Build # 11313 - Failure!
Date Tue, 23 Sep 2014 09:03:51 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11313/
Java: 32bit/jdk1.8.0_20 -client -XX:+UseParallelGC

2 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1
	at __randomizedtesting.SeedInfo.seed([8601EDE3FBF79870:7E763FB8CA8F84C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:583)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.testDistribSearch

Error Message:
No live SolrServers available to handle this request:[https://127.0.0.1:45902/nw, https://127.0.0.1:57567/nw, https://127.0.0.1:33972/nw]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:45902/nw, https://127.0.0.1:57567/nw, https://127.0.0.1:33972/nw]
	at __randomizedtesting.SeedInfo.seed([8601EDE3FBF79870:7E763FB8CA8F84C]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:322)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:880)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
	at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.removeAndWaitForLastReplicaGone(DeleteLastCustomShardedReplicaTest.java:117)
	at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.doTest(DeleteLastCustomShardedReplicaTest.java:107)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not  remove replica : customcollreplicadeletion/a/core_node2
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304)
	... 45 more




Build Log:
[...truncated 11522 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/init-core-data-001
   [junit4]   2> 1122599 T2482 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1122599 T2482 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 1122606 T2482 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 1122606 T2482 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1122607 T2483 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1122707 T2482 oasc.ZkTestServer.run start zk server on port:47586
   [junit4]   2> 1122708 T2482 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1122709 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1122714 T2489 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1738417 name:ZooKeeperConnection Watcher:127.0.0.1:47586 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1122715 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1122716 T2482 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1122716 T2482 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1122722 T2482 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1122723 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1122725 T2491 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28c35e name:ZooKeeperConnection Watcher:127.0.0.1:47586/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1122725 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1122726 T2482 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1122726 T2482 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1122732 T2482 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1122737 T2482 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1122740 T2482 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1122743 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1122743 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1122747 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1122748 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1122750 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1122751 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1122754 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1122754 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1122756 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1122757 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1122759 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1122760 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 1122762 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1122763 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1122765 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1122766 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1122768 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1122769 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1122771 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1122772 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1122774 T2482 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1122775 T2482 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> Using new style solr.xml
   [junit4]   2> 1123109 T2482 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1123113 T2482 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47187
   [junit4]   2> 1123113 T2482 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1123113 T2482 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1123114 T2482 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002
   [junit4]   2> 1123114 T2482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/'
   [junit4]   2> 1123138 T2482 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/solr.xml
   [junit4]   2> 1123149 T2482 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002
   [junit4]   2> 1123150 T2482 oasc.CoreContainer.<init> New CoreContainer 7677857
   [junit4]   2> 1123150 T2482 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/]
   [junit4]   2> 1123151 T2482 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1123151 T2482 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1123152 T2482 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1123152 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1123152 T2482 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1123153 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1123153 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1123153 T2482 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1123154 T2482 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1123154 T2482 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1123155 T2482 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1123155 T2482 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1123155 T2482 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1123156 T2482 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47586/solr
   [junit4]   2> 1123156 T2482 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1123156 T2482 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1123157 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1123158 T2502 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fdc504 name:ZooKeeperConnection Watcher:127.0.0.1:47586 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1123159 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1123159 T2482 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1123161 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1123162 T2504 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1be4026 name:ZooKeeperConnection Watcher:127.0.0.1:47586/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1123162 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1123164 T2482 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1123166 T2482 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 1123169 T2482 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 1123172 T2482 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 1123174 T2482 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 1123177 T2482 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 1123179 T2482 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47187_
   [junit4]   2> 1123179 T2482 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47187_
   [junit4]   2> 1123184 T2482 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 1123188 T2482 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 1123192 T2482 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1123197 T2482 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:47187_
   [junit4]   2> 1123198 T2482 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1123202 T2482 oasc.Overseer.start Overseer (id=92501499578941443-127.0.0.1:47187_-n_0000000000) starting
   [junit4]   2> 1123208 T2482 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 1123225 T2482 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1123226 T2506 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1123227 T2482 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1123231 T2482 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1123235 T2482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1123244 T2505 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1123248 T2482 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002
   [junit4]   2> 1123260 T2482 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/conf/
   [junit4]   2> 1123408 T2482 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1123410 T2482 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 1123411 T2482 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1123411 T2482 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1123412 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1123414 T2510 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d903b name:ZooKeeperConnection Watcher:127.0.0.1:47586/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1123415 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1123415 T2482 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1123417 T2482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1123425 T2482 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1123425 T2482 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
   [junit4]   2> 1123426 T2482 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1123427 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1123428 T2512 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@172fe34 name:ZooKeeperConnection Watcher:127.0.0.1:47586/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1123429 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1123429 T2482 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1123433 T2513 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1123438 T2505 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: collection1
   [junit4]   2> 1123439 T2505 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 1123439 T2505 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2
   [junit4]   2> 1123444 T2505 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json
   [junit4]   2> 1123446 T2513 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1123446 T2514 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1123758 T2482 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 1123759 T2482 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1123761 T2482 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34567
   [junit4]   2> 1123762 T2482 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1123762 T2482 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1123762 T2482 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003
   [junit4]   2> 1123763 T2482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/'
   [junit4]   2> 1123783 T2482 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/solr.xml
   [junit4]   2> 1123794 T2482 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003
   [junit4]   2> 1123795 T2482 oasc.CoreContainer.<init> New CoreContainer 9051148
   [junit4]   2> 1123795 T2482 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/]
   [junit4]   2> 1123796 T2482 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1123796 T2482 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1123796 T2482 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1123797 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1123797 T2482 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1123797 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1123798 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1123798 T2482 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1123798 T2482 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1123799 T2482 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1123799 T2482 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1123800 T2482 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1123800 T2482 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1123800 T2482 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47586/solr
   [junit4]   2> 1123801 T2482 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1123801 T2482 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1123802 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1123803 T2525 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@131a5f6 name:ZooKeeperConnection Watcher:127.0.0.1:47586 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1123804 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1123804 T2482 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1123806 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1123806 T2527 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cc9355 name:ZooKeeperConnection Watcher:127.0.0.1:47586/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1123807 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1123812 T2482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1124819 T2482 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34567_
   [junit4]   2> 1124821 T2482 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34567_
   [junit4]   2> 1124830 T2482 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1124836 T2482 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003
   [junit4]   2> 1124843 T2482 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/conf/
   [junit4]   2> 1124979 T2482 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1124980 T2482 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 1124980 T2482 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1125255 T2482 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 1125255 T2482 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1125257 T2482 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34621
   [junit4]   2> 1125258 T2482 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1125258 T2482 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1125259 T2482 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-004
   [junit4]   2> 1125259 T2482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-004/'
   [junit4]   2> 1125277 T2482 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-004/solr.xml
   [junit4]   2> 1125289 T2482 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-004
   [junit4]   2> 1125290 T2482 oasc.CoreContainer.<init> New CoreContainer 5239361
   [junit4]   2> 1125290 T2482 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-004/]
   [junit4]   2> 1125291 T2482 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1125291 T2482 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1125291 T2482 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1125292 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1125292 T2482 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1125292 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1125292 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1125293 T2482 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1125293 T2482 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1125294 T2482 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1125294 T2482 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1125294 T2482 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1125295 T2482 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1125295 T2482 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47586/solr
   [junit4]   2> 1125295 T2482 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1125296 T2482 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1125296 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1125298 T2540 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e0cea name:ZooKeeperConnection Watcher:127.0.0.1:47586 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1125298 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1125299 T2482 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1125301 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1125303 T2542 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e58c6a name:ZooKeeperConnection Watcher:127.0.0.1:47586/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1125303 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1125316 T2482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1126323 T2482 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34621_
   [junit4]   2> 1126325 T2482 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34621_
   [junit4]   2> 1126334 T2482 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1126339 T2482 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-004
   [junit4]   2> 1126345 T2482 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-004/conf/
   [junit4]   2> 1126471 T2482 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1126473 T2482 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 1126473 T2482 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1126745 T2482 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 1126746 T2482 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1126748 T2482 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:32970
   [junit4]   2> 1126749 T2482 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1126750 T2482 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1126750 T2482 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005
   [junit4]   2> 1126750 T2482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/'
   [junit4]   2> 1126775 T2482 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/solr.xml
   [junit4]   2> 1126789 T2482 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005
   [junit4]   2> 1126790 T2482 oasc.CoreContainer.<init> New CoreContainer 30491368
   [junit4]   2> 1126790 T2482 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/]
   [junit4]   2> 1126791 T2482 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1126791 T2482 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1126791 T2482 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1126792 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1126792 T2482 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1126792 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1126793 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1126793 T2482 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1126793 T2482 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1126794 T2482 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1126794 T2482 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1126795 T2482 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1126795 T2482 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1126795 T2482 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47586/solr
   [junit4]   2> 1126796 T2482 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1126796 T2482 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1126797 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1126798 T2555 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a5b12e name:ZooKeeperConnection Watcher:127.0.0.1:47586 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1126799 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1126800 T2482 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1126803 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1126804 T2557 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69537b name:ZooKeeperConnection Watcher:127.0.0.1:47586/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1126805 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1126816 T2482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1127825 T2482 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32970_
   [junit4]   2> 1127827 T2482 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32970_
   [junit4]   2> 1127836 T2482 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1127841 T2482 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005
   [junit4]   2> 1127855 T2482 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/conf/
   [junit4]   2> 1128040 T2482 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1128041 T2482 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 1128042 T2482 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1128334 T2482 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 1128335 T2482 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1128337 T2482 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47245
   [junit4]   2> 1128338 T2482 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1128338 T2482 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1128338 T2482 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-006
   [junit4]   2> 1128339 T2482 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-006/'
   [junit4]   2> 1128379 T2482 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-006/solr.xml
   [junit4]   2> 1128397 T2482 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-006
   [junit4]   2> 1128398 T2482 oasc.CoreContainer.<init> New CoreContainer 12588938
   [junit4]   2> 1128398 T2482 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-006/]
   [junit4]   2> 1128399 T2482 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1128399 T2482 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1128400 T2482 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1128400 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1128400 T2482 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1128400 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1128401 T2482 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1128401 T2482 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1128401 T2482 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1128402 T2482 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1128402 T2482 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1128403 T2482 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1128403 T2482 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1128403 T2482 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47586/solr
   [junit4]   2> 1128403 T2482 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1128404 T2482 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1128405 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1128406 T2570 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@187ce5e name:ZooKeeperConnection Watcher:127.0.0.1:47586 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1128407 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1128408 T2482 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1128410 T2482 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1128411 T2572 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c41c6c name:ZooKeeperConnection Watcher:127.0.0.1:47586/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1128411 T2482 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1128422 T2482 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1129430 T2482 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47245_
   [junit4]   2> 1129432 T2482 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47245_
   [junit4]   2> 1129440 T2482 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1129444 T2482 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-006
   [junit4]   2> 1129455 T2482 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-006/conf/
   [junit4]   2> 1129629 T2482 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1129630 T2482 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 1129631 T2482 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1129637 T2523 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&collection.configName=conf1&router.field=myOwnField&name=solrj_collection&action=CREATE&numShards=2&wt=javabin&version=2
   [junit4]   2> 1129640 T2513 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1129642 T2506 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField"}
   [junit4]   2> 1129643 T2575 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField"}
   [junit4]   2> 1129645 T2575 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection 
   [junit4]   2> 1129646 T2575 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
   [junit4]   2> 1129653 T2513 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1129658 T2505 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
   [junit4]   2> 1129658 T2505 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
   [junit4]   2> 1129659 T2505 oasc.Overseer$ClusterStateUpdater.createCollection state version solrj_collection 1
   [junit4]   2> 1129661 T2543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129661 T2558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129662 T2573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129661 T2528 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129662 T2513 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129662 T2514 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129753 T2575 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection solrj_collection, shardNames [shard1, shard2] , replicationFactor : 2
   [junit4]   2> 1129754 T2575 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:34621_
   [junit4]   2> 1129755 T2575 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:32970_
   [junit4]   2> 1129757 T2575 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:47187_
   [junit4]   2> 1129759 T2575 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:34567_
   [junit4]   2> 1129760 T2535 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
   [junit4]   2> 1129762 T2499 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
   [junit4]   2> 1129764 T2550 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
   [junit4]   2> 1129765 T2520 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
   [junit4]   2> 1129766 T2535 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection
   [junit4]   2> 1129767 T2550 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection
   [junit4]   2> 1129767 T2499 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection
   [junit4]   2> 1129768 T2513 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1129769 T2535 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1129768 T2520 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection
   [junit4]   2> 1129770 T2499 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1129769 T2550 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1129771 T2520 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1129776 T2505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:34621",
   [junit4]   2> 	  "node_name":"127.0.0.1:34621_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1129780 T2505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:32970",
   [junit4]   2> 	  "node_name":"127.0.0.1:32970_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1129781 T2513 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1129782 T2505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:47187",
   [junit4]   2> 	  "node_name":"127.0.0.1:47187_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1129785 T2505 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:34567",
   [junit4]   2> 	  "node_name":"127.0.0.1:34567_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 1129787 T2513 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1129788 T2543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129788 T2514 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129788 T2513 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129788 T2528 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129788 T2573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1129788 T2558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1130769 T2535 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
   [junit4]   2> 1130770 T2535 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 1130771 T2499 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
   [junit4]   2> 1130771 T2550 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
   [junit4]   2> 1130772 T2535 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1130773 T2535 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 1130772 T2499 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 1130772 T2520 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
   [junit4]   2> 1130775 T2499 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1130776 T2499 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 1130772 T2550 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 1130775 T2520 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 1130778 T2550 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1130778 T2550 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 1130775 T2535 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1130778 T2520 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1130780 T2520 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 1130777 T2499 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1130780 T2550 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1130779 T2535 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-004/solrj_collection_shard1_replica1/'
   [junit4]   2> 1130782 T2550 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/solrj_collection_shard1_replica2/'
   [junit4]   2> 1130781 T2520 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1130781 T2499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/solrj_collection_shard2_replica1/'
   [junit4]   2> 1130784 T2520 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/solrj_collection_shard2_replica2/'
   [junit4]   2> 1130851 T2535 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1130852 T2550 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1130854 T2499 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1130863 T2520 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1130885 T2550 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1130886 T2499 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1130887 T2550 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1130889 T2499 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1130891 T2535 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1130893 T2535 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1130900 T2520 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1130901 T2520 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1130935 T2550 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
   [junit4]   2> 1130935 T2520 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
   [junit4]   2> 1130938 T2499 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
   [junit4]   2> 1130942 T2535 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
   [junit4]   2> 1131084 T2520 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1131091 T2520 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1131094 T2520 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1131119 T2499 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1131119 T2520 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1131126 T2520 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1131129 T2499 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1131130 T2520 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1131131 T2499 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1131132 T2520 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1131136 T2520 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1131137 T2520 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1131138 T2520 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1131138 T2520 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1131139 T2550 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1131139 T2520 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using configuration from collection solrj_collection
   [junit4]   2> 1131140 T2520 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1131140 T2520 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/solrj_collection_shard2_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/solrj_collection_shard2_replica2/data/
   [junit4]   2> 1131141 T2520 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f1b296
   [junit4]   2> 1131144 T2520 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/solrj_collection_shard2_replica2/data
   [junit4]   2> 1131145 T2520 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/solrj_collection_shard2_replica2/data/index/
   [junit4]   2> 1131145 T2520 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1131146 T2520 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-003/solrj_collection_shard2_replica2/data/index
   [junit4]   2> 1131147 T2520 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1689189644, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1131149 T2520 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-52323b)),segFN=segments_1,generation=1}
   [junit4]   2> 1131149 T2520 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1131150 T2550 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1131152 T2550 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1131157 T2535 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1131157 T2520 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1131158 T2520 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1131159 T2520 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1131159 T2520 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1131159 T2520 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1131160 T2520 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1131160 T2520 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1131161 T2499 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1131161 T2520 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1131161 T2520 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1131162 T2520 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1131163 T2520 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1131164 T2520 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1131164 T2535 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1131165 T2499 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1131165 T2520 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1131167 T2535 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1131167 T2520 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1131167 T2520 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1131168 T2499 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1131168 T2520 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1131170 T2520 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1131170 T2499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1131171 T2499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1131171 T2520 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1131172 T2499 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1131173 T2520 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1131173 T2499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1131174 T2499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1131174 T2520 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 1131174 T2499 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using configuration from collection solrj_collection
   [junit4]   2> 1131175 T2499 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1131176 T2499 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/solrj_collection_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/solrj_collection_shard2_replica1/data/
   [junit4]   2> 1131177 T2499 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f1b296
   [junit4]   2> 1131176 T2550 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1131177 T2520 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1131178 T2499 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/solrj_collection_shard2_replica1/data
   [junit4]   2> 1131179 T2499 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/solrj_collection_shard2_replica1/data/index/
   [junit4]   2> 1131179 T2499 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1131180 T2520 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1131180 T2550 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1131181 T2499 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-002/solrj_collection_shard2_replica1/data/index
   [junit4]   2> 1131182 T2499 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1689189644, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1131183 T2520 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1131183 T2499 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-32b176)),segFN=segments_1,generation=1}
   [junit4]   2> 1131184 T2499 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1131184 T2550 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1131185 T2520 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1131185 T2550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1131186 T2550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1131187 T2550 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1131188 T2550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1131188 T2520 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1131188 T2550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1131189 T2520 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1131189 T2550 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using configuration from collection solrj_collection
   [junit4]   2> 1131190 T2550 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1131191 T2550 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/solrj_collection_shard1_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/solrj_collection_shard1_replica2/data/
   [junit4]   2> 1131191 T2520 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=65.2333984375, floorSegmentMB=2.1748046875, forceMergeDeletesPctAllowed=27.712847795871962, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4344243477098072
   [junit4]   2> 1131191 T2499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1131191 T2535 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1131192 T2499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1131191 T2550 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1f1b296
   [junit4]   2> 1131193 T2499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1131192 T2520 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-52323b)),segFN=segments_1,generation=1}
   [junit4]   2> 1131194 T2550 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/solrj_collection_shard1_replica2/data
   [junit4]   2> 1131193 T2499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1131194 T2550 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/solrj_collection_shard1_replica2/data/index/
   [junit4]   2> 1131194 T2520 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1131195 T2550 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1131196 T2535 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1131195 T2499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1131196 T2550 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-8601EDE3FBF79870-001/tempDir-005/solrj_collection_shard1_replica2/data/index
   [junit4]   2> 1131197 T2499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1131197 T2520 oass.SolrIndexSearcher.<init> Opening Searcher@5fdc5a[solrj_collection_shard2_replica2] main
   [junit4]   2> 1131197 T2550 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1689189644, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1131198 T2520 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 1131199 T2535 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1131198 T2499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1131199 T2550 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-139125a)),segFN=segments_1,generation=1}
   [junit4]   2> 1131200 T2520 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1131200 T2499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1131201 T2499 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1131200 T2520 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1131200 T2535 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1131202 T2535 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1131200 T2550 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1131202 T2535 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1131202 T2499 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1131201 T2520 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1131204 T2520 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1131204 T2535 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1131205 T2535 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1131205 T2520 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1131205 T2499 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 1131206 T2520 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1131207 T2520 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1131206 T2535 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using c

[...truncated too long message...]

java:263)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> 2545649 T7011 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher  java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7472b5 rejected from java.util.concurrent.ThreadPoolExecutor@5b8dfa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 4]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:263)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=DeleteLastCustomShardedReplicaTest -Dtests.method=testDistribSearch -Dtests.seed=8601EDE3FBF79870 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=zh_TW -Dtests.timezone=Asia/Dhaka -Dtests.file.encoding=UTF-8
   [junit4] ERROR    131s J0 | DeleteLastCustomShardedReplicaTest.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:45902/nw, https://127.0.0.1:57567/nw, https://127.0.0.1:33972/nw]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([8601EDE3FBF79870:7E763FB8CA8F84C]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:322)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:880)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
   [junit4]    > 	at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.removeAndWaitForLastReplicaGone(DeleteLastCustomShardedReplicaTest.java:117)
   [junit4]    > 	at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.doTest(DeleteLastCustomShardedReplicaTest.java:107)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not  remove replica : customcollreplicadeletion/a/core_node2
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304)
   [junit4]    > 	... 45 more
   [junit4]   2> 2545684 T6945 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteLastCustomShardedReplicaTest-8601EDE3FBF79870-001
   [junit4]   2> 131426 T6944 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene410): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=zh_TW, timezone=Asia/Dhaka
   [junit4]   2> NOTE: Linux 3.13.0-35-generic i386/Oracle Corporation 1.8.0_20 (32-bit)/cpus=8,threads=1,free=211011520,total=357564416
   [junit4]   2> NOTE: All tests run in this JVM: [SuggesterWFSTTest, TimeZoneUtilsTest, TestSolrCoreProperties, TestFastOutputStream, DocValuesMultiTest, TestFastWriter, SuggestComponentTest, TestNonDefinedSimilarityFactory, TestManagedSchemaFieldResource, HdfsWriteToMultipleCollectionsTest, DirectSolrSpellCheckerTest, DistributedSpellCheckComponentTest, LeaderElectionTest, PingRequestHandlerTest, TestStandardQParsers, TestDefaultSimilarityFactory, TriLevelCompositeIdRoutingTest, PathHierarchyTokenizerFactoryTest, TestSolrDeletionPolicy1, TestWordDelimiterFilterFactory, IndexBasedSpellCheckerTest, TestChildDocTransformer, TestCodecSupport, RequiredFieldsTest, JsonLoaderTest, DistributedQueryComponentOptimizationTest, TestUniqueKeyFieldResource, BadCopyFieldTest, TestReload, TestInitParams, TestReplicationHandler, TestLuceneMatchVersion, TestBinaryResponseWriter, DirectUpdateHandlerTest, TestClassNameShortening, TestTrie, SuggesterTSTTest, VMParamsZkACLAndCredentialsProvidersTest, ClusterStateUpdateTest, TestPartialUpdateDeduplication, TestFieldCollectionResource, HighlighterTest, TestHashPartitioner, TestTolerantSearch, ResponseLogComponentTest, TestRestManager, DisMaxRequestHandlerTest, ReplicationFactorTest, TestRangeQuery, TestFileDictionaryLookup, SSLMigrationTest, MBeansHandlerTest, HdfsCollectionsAPIDistributedZkTest, TestCloudManagedSchema, PluginInfoTest, TestCoreDiscovery, MigrateRouteKeyTest, TestFastLRUCache, FieldMutatingUpdateProcessorTest, TestSearchPerf, WordBreakSolrSpellCheckerTest, TestFuzzyAnalyzedSuggestions, TestSolrXmlPersistence, ShowFileRequestHandlerTest, ZkControllerTest, ConnectionManagerTest, AtomicUpdatesTest, CoreAdminHandlerTest, TestDFRSimilarityFactory, SpellingQueryConverterTest, TestSolr4Spatial, SuggesterTest, SolrCmdDistributorTest, DocumentAnalysisRequestHandlerTest, TestDocSet, RollingRestartTest, TestSystemIdResolver, TestNoOpRegenerator, OpenExchangeRatesOrgProviderTest, DocExpirationUpdateProcessorFactoryTest, TestQuerySenderListener, QueryEqualityTest, TestSuggestSpellingConverter, TestStressRecovery, MinimalSchemaTest, TestFaceting, TestRTGBase, TestFoldingMultitermQuery, DistribDocExpirationUpdateProcessorTest, BasicFunctionalityTest, TestImplicitCoreProperties, TestDynamicFieldResource, TestComplexPhraseQParserPlugin, ZkNodePropsTest, SystemInfoHandlerTest, TestSolrQueryParserDefaultOperatorResource, MoreLikeThisHandlerTest, TestFieldTypeResource, InfoHandlerTest, TestRemoteStreaming, DistributedSuggestComponentTest, TestStressUserVersions, XsltUpdateRequestHandlerTest, TestDynamicFieldCollectionResource, OverseerCollectionProcessorTest, BasicDistributedZkTest, UnloadDistributedZkTest, SyncSliceTest, OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, RecoveryZkTest, TestZkChroot, TestRandomDVFaceting, ZkCLITest, TestDistributedSearch, TestDistributedGrouping, TestRealTimeGet, TestStressReorder, TestStressVersions, HardAutoCommitTest, SimpleFacetsTest, SolrCoreTest, SpellCheckComponentTest, TestGroupingSearch, PeerSyncTest, SoftAutoCommitTest, SuggesterFSTTest, SpatialFilterTest, PolyFieldTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, DirectUpdateHandlerOptimizeTest, StatelessScriptUpdateProcessorFactoryTest, TestSurroundQueryParser, TestQueryUtils, StandardRequestHandlerTest, TestWriterPerf, PrimitiveFieldTypeTest, TestOmitPositions, XmlUpdateRequestHandlerTest, TestValueSourceCache, TermVectorComponentTest, TestArbitraryIndexDir, LoggingHandlerTest, IndexSchemaTest, TestCollationField, ReturnFieldsTest, TestCSVResponseWriter, CSVRequestHandlerTest, SearchHandlerTest, HighlighterConfigTest, TestSolrIndexConfig, CopyFieldTest, BadComponentTest, TestSolrDeletionPolicy2, MultiTermTest, TestConfig, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, SpellPossibilityIteratorTest, SynonymTokenizerTest, EchoParamsTest, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, URLClassifyProcessorTest, ChaosMonkeyNothingIsSafeTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, AliasIntegrationTest, AsyncMigrateRouteKeyTest, DeleteLastCustomShardedReplicaTest]
   [junit4] Completed on J0 in 131.72s, 1 test, 1 error <<< FAILURES!

[...truncated 280 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:471: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:60: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1358: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:961: There were test failures: 429 suites, 1774 tests, 2 errors, 45 ignored (23 assumptions)

Total time: 144 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_20 -client -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Mime
View raw message