lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0) - Build # 9843 - Still Failing!
Date Wed, 19 Mar 2014 07:45:42 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9843/
Java: 32bit/jdk1.8.0 -server -XX:+UseConcMarkSweepGC

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestModifyConfFiles

Error Message:
ERROR: SolrZkClient opens=4 closes=3

Stack Trace:
java.lang.AssertionError: ERROR: SolrZkClient opens=4 closes=3
	at __randomizedtesting.SeedInfo.seed([229540E9FC128DC3]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:435)
	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:180)
	at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:789)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
	at java.lang.Thread.run(Thread.java:744)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestModifyConfFiles

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestModifyConfFiles:     1) Thread[id=7319, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-EventThread, state=WAITING, group=TGRP-TestModifyConfFiles]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)    2) Thread[id=7318, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-SendThread(localhost.localdomain:59210), state=TIMED_WAITING, group=TGRP-TestModifyConfFiles]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestModifyConfFiles: 
   1) Thread[id=7319, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-EventThread, state=WAITING, group=TGRP-TestModifyConfFiles]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   2) Thread[id=7318, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-SendThread(localhost.localdomain:59210), state=TIMED_WAITING, group=TGRP-TestModifyConfFiles]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
	at __randomizedtesting.SeedInfo.seed([229540E9FC128DC3]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestModifyConfFiles

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=7318, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-SendThread(localhost.localdomain:59210), state=TIMED_WAITING, group=TGRP-TestModifyConfFiles]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=7318, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-SendThread(localhost.localdomain:59210), state=TIMED_WAITING, group=TGRP-TestModifyConfFiles]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
	at __randomizedtesting.SeedInfo.seed([229540E9FC128DC3]:0)


REGRESSION:  org.apache.solr.cloud.TestModifyConfFiles.testDistribSearch

Error Message:


Stack Trace:
org.apache.solr.common.cloud.ZooKeeperException: 
	at __randomizedtesting.SeedInfo.seed([229540E9FC128DC3:A373CEF18B4DEDFF]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:249)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.initCloud(AbstractFullDistribZkTestBase.java:256)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:312)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:868)
	at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:826)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:862)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
	at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /live_nodes
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:299)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:242)
	... 41 more




Build Log:
[...truncated 11235 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestModifyConfFiles
   [junit4]   2> 1500036 T7297 oas.SolrTestCaseJ4.startTrackingSearchers WARN startTrackingSearchers: numOpens=6 numCloses=6
   [junit4]   2> 1500036 T7297 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1500037 T7297 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /juzl/oi
   [junit4]   2> 1500040 T7297 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-TestModifyConfFiles-1395215080199
   [junit4]   2> 1500040 T7297 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1500041 T7298 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1500141 T7297 oasc.ZkTestServer.run start zk server on port:59210
   [junit4]   2> 1500142 T7297 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1500145 T7304 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1047487 name:ZooKeeperConnection Watcher:127.0.0.1:59210 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1500146 T7297 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1500146 T7297 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1500152 T7297 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1500153 T7306 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@145de0d name:ZooKeeperConnection Watcher:127.0.0.1:59210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1500153 T7297 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1500153 T7297 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1500157 T7297 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1500161 T7297 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1500165 T7297 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1500170 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1500170 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1500176 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1500177 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1500182 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1500182 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1500186 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1500187 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1500191 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1500192 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1500196 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1500197 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 1500200 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1500201 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1500203 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1500203 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1500205 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1500205 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1500207 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1500208 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1500212 T7297 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1500213 T7297 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1500433 T7297 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1500437 T7297 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60519
   [junit4]   2> 1500437 T7297 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1500438 T7297 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1500439 T7297 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.TestModifyConfFiles-controljetty-1395215080377
   [junit4]   2> 1500439 T7297 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.TestModifyConfFiles-controljetty-1395215080377/'
   [junit4]   2> 1500451 T7297 oass.SolrDispatchFilter.loadConfigSolr Trying to read solr.xml from 127.0.0.1:59210/solr
   [junit4]   2> 1500452 T7297 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1500453 T7317 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@133daa5 name:ZooKeeperConnection Watcher:127.0.0.1:59210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1500453 T7297 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1500456 T7297 oass.SolrDispatchFilter.init ERROR Could not start Solr. Check solr/home property and the logs
   [junit4]   2> 1500457 T7297 oasc.SolrException.log ERROR null:org.apache.solr.common.SolrException: Could not load solr.xml from zookeeper
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.loadConfigSolr(SolrDispatchFilter.java:182)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:198)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
   [junit4]   2> 		at org.eclipse.jetty.servlet.FilterHolder.doStart(FilterHolder.java:119)
   [junit4]   2> 		at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:719)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1309)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1345)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1085)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:931)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:888)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:340)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:325)
   [junit4]   2> 		at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:174)
   [junit4]   2> 		at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:65)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:430)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:403)
   [junit4]   2> 		at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJetty(AbstractFullDistribZkTestBase.java:490)
   [junit4]   2> 		at org.apache.solr.BaseDistributedSearchTestCase.createJetty(BaseDistributedSearchTestCase.java:356)
   [junit4]   2> 		at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:291)
   [junit4]   2> 		at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:868)
   [junit4]   2> 		at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
   [junit4]   2> 		at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 		at java.lang.reflect.Method.invoke(Method.java:483)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:826)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:862)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 		at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
   [junit4]   2> 		at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 		at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 	Caused by: org.apache.solr.common.SolrException: Could not load solr.xml from zookeeper: node not found
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.loadConfigSolr(SolrDispatchFilter.java:178)
   [junit4]   2> 		... 58 more
   [junit4]   2> 	
   [junit4]   2> 1500458 T7297 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1500460 T7297 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1500461 T7319 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1130eb3 name:ZooKeeperConnection Watcher:127.0.0.1:59210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1500462 T7297 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1500463 T7297 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1500466 T7297 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1500469 T7297 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1500471 T7297 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 60519
   [junit4]   2> 1500493 T7297 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/juzl/oi,null}
   [junit4]   2> 1500547 T7297 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 1500550 T7297 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:59210 59210
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestModifyConfFiles -Dtests.method=testDistribSearch -Dtests.seed=229540E9FC128DC3 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ko -Dtests.timezone=GB-Eire -Dtests.file.encoding=UTF-8
   [junit4] ERROR   10.7s J0 | TestModifyConfFiles.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.common.cloud.ZooKeeperException: 
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([229540E9FC128DC3:A373CEF18B4DEDFF]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:249)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.initCloud(AbstractFullDistribZkTestBase.java:256)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:312)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:868)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]    > Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /live_nodes
   [junit4]    > 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
   [junit4]    > 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]    > 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]    > 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
   [junit4]    > 	at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:299)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:242)
   [junit4]    > 	... 41 more
   [junit4]   2> 1510766 T7297 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 1510767 T7297 oas.SolrTestCaseJ4.endTrackingZkClients ERROR ERROR: SolrZkClient opens=4 closes=3
   [junit4]   2> 10734 T7296 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> 1510844 T7319 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1130eb3 name:ZooKeeperConnection Watcher:127.0.0.1:59210/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
   [junit4]   2> 1510845 T7319 oascc.ConnectionManager.process zkClient has disconnected
   [junit4]   2> 1512272 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1514033 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1515752 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1517804 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1519680 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1521055 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1522876 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1524290 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1525396 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1527099 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1528249 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1529396 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 30870 T7296 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestModifyConfFiles: 
   [junit4]   2> 	   1) Thread[id=7319, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-EventThread, state=WAITING, group=TGRP-TestModifyConfFiles]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 	   2) Thread[id=7318, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-SendThread(localhost.localdomain:59210), state=TIMED_WAITING, group=TGRP-TestModifyConfFiles]
   [junit4]   2> 	        at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> 30871 T7296 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=7319, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-EventThread, state=WAITING, group=TGRP-TestModifyConfFiles]
   [junit4]   2> 	   2) Thread[id=7318, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-SendThread(localhost.localdomain:59210), state=TIMED_WAITING, group=TGRP-TestModifyConfFiles]
   [junit4]   2> 1530908 T7318 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> 
   [junit4]   2> 1530907 T7319 oaz.ClientCnxn$EventThread.run ERROR Event thread exiting due to interruption java.lang.InterruptedException
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
   [junit4]   2> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
   [junit4]   2> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]   2> 
   [junit4]   2> 1530909 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1531407 T7318 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> 
   [junit4]   2> 1531408 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1531908 T7318 oaz.ClientCnxn$SendThread.run WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984)
   [junit4]   2> 
   [junit4]   2> 1532408 T7318 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> 
   [junit4]   2> 1532409 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 1532908 T7318 oaz.ClientCnxn$SendThread.run WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:984)
   [junit4]   2> 
   [junit4]   2> 1533409 T7318 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 	at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> 
   [junit4]   2> 1533410 T7318 oaz.ClientCnxn$SendThread.run WARN Session 0x144d94c9b0a0003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 
   [junit4]   2> 33873 T7296 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
   [junit4]   2> 	   1) Thread[id=7318, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-SendThread(localhost.localdomain:59210), state=TIMED_WAITING, group=TGRP-TestModifyConfFiles]
   [junit4]   2> 	        at java.lang.Thread.sleep(Native Method)
   [junit4]   2> 	        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]   2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=ko, timezone=GB-Eire
   [junit4]   2> NOTE: Linux 3.8.0-37-generic i386/Oracle Corporation 1.8.0 (32-bit)/cpus=8,threads=2,free=57564480,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [CachingDirectoryFactoryTest, CSVRequestHandlerTest, TestFieldTypeResource, DOMUtilTest, TestDocumentBuilder, TestFieldResource, DistributedTermsComponentTest, SampleTest, TestComplexPhraseQParserPlugin, BinaryUpdateRequestHandlerTest, SOLR749Test, TestLuceneMatchVersion, QueryElevationComponentTest, TestPostingsSolrHighlighter, TriLevelCompositeIdRoutingTest, BlockCacheTest, InfoHandlerTest, FunctionTest, BlockDirectoryTest, TermsComponentTest, TestExceedMaxTermLength, TestSolrXMLSerializer, TestNonNRTOpen, TestSchemaSimilarityResource, LoggingHandlerTest, TestOverriddenPrefixQueryForCustomFieldType, StatelessScriptUpdateProcessorFactoryTest, TestRecoveryHdfs, TestLMJelinekMercerSimilarityFactory, TestQueryUtils, ExternalFileFieldSortTest, TestPHPSerializedResponseWriter, TestInitQParser, TestFunctionQuery, DeleteInactiveReplicaTest, BadComponentTest, DocValuesMissingTest, AnalysisAfterCoreReloadTest, TestPerFieldSimilarity, TestCollationFieldDocValues, DistributedSpellCheckComponentTest, AbstractAnalyticsStatsTest, TestSuggestSpellingConverter, TestGroupingSearch, DistributedQueryElevationComponentTest, NoFacetTest, JsonLoaderTest, TestCoreContainer, ClusterStateUpdateTest, HdfsDirectoryTest, OverseerTest, TermVectorComponentTest, SolrTestCaseJ4Test, TestFreeTextSuggestions, SuggesterFSTTest, TestReplicationHandler, LukeRequestHandlerTest, TestConfig, SoftAutoCommitTest, ZkCLITest, SuggestComponentTest, SolrIndexSplitterTest, DistributedExpandComponentTest, FastVectorHighlighterTest, FieldMutatingUpdateProcessorTest, TestDistributedMissingSort, DeleteShardTest, ModifyConfFileTest, TestSolrIndexConfig, EnumFieldTest, BasicFunctionalityTest, TestSearchPerf, SchemaVersionSpecificBehaviorTest, URLClassifyProcessorTest, TestSolrQueryParser, TestCloudManagedSchemaAddField, SSLMigrationTest, TestExtendedDismaxParser, CacheHeaderTest, MoreLikeThisHandlerTest, FieldFacetExtrasTest, ClusterStateTest, TestAddFieldRealTimeGet, SystemInfoHandlerTest, SearchHandlerTest, LeaderElectionTest, RemoteQueryErrorTest, CursorPagingTest, DirectUpdateHandlerOptimizeTest, QueryParsingTest, StressHdfsTest, TestElisionMultitermQuery, CopyFieldTest, SyncSliceTest, TestStressVersions, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest, TestRandomDVFaceting, SliceStateUpdateTest, TestDistributedGrouping, TermVectorComponentDistributedTest, TestRealTimeGet, TestJoin, SimpleFacetsTest, StatsComponentTest, SolrCmdDistributorTest, ConvertedLegacyTest, TestLazyCores, TestBadConfig, TestIndexSearcher, ShowFileRequestHandlerTest, TestCoreDiscovery, DocValuesTest, SuggesterTSTTest, SuggesterTest, SpatialFilterTest, TestCSVLoader, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, TestAtomicUpdateErrorCases, QueryEqualityTest, TestRemoteStreaming, TestSolrDeletionPolicy1, IndexBasedSpellCheckerTest, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, PrimitiveFieldTypeTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, PathHierarchyTokenizerFactoryTest, RequiredFieldsTest, TestArbitraryIndexDir, IndexSchemaTest, TestCollationField, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, TestComponentsName, CoreContainerCoreInitFailuresTest, HighlighterConfigTest, UpdateParamsTest, ResponseLogComponentTest, TestMergePolicyConfig, MultiTermTest, TestDocSet, TestBinaryField, NumericFieldsTest, MinimalSchemaTest, TestSolrCoreProperties, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, EchoParamsTest, TestIBSimilarityFactory, TestDefaultSimilarityFactory, LegacyHTMLStripCharFilterTest, TimeZoneUtilsTest, ResourceLoaderTest, ScriptEngineTest, DateMathParserTest, PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, DateFieldTest, TestLRUCache, TestUtils, ZkNodePropsTest, SliceStateTest, UUIDFieldTest, CircularListTest, TestCursorMarkWithoutUniqueKey, QueryFacetTest, AliasIntegrationTest, AssignTest, AsyncMigrateRouteKeyTest, ConnectionManagerTest, CustomCollectionTest, DeleteReplicaTest, DistribCursorPagingTest, MigrateRouteKeyTest, OverseerRolesTest, TestDistribDocBasedVersion, TestModifyConfFiles]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestModifyConfFiles -Dtests.seed=229540E9FC128DC3 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ko -Dtests.timezone=GB-Eire -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | TestModifyConfFiles (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ERROR: SolrZkClient opens=4 closes=3
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([229540E9FC128DC3]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:435)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:180)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestModifyConfFiles: 
   [junit4]    >    1) Thread[id=7319, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-EventThread, state=WAITING, group=TGRP-TestModifyConfFiles]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
   [junit4]    >    2) Thread[id=7318, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-SendThread(localhost.localdomain:59210), state=TIMED_WAITING, group=TGRP-TestModifyConfFiles]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([229540E9FC128DC3]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=7318, name=TEST-TestModifyConfFiles.testDistribSearch-seed#[229540E9FC128DC3]-SendThread(localhost.localdomain:59210), state=TIMED_WAITING, group=TGRP-TestModifyConfFiles]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([229540E9FC128DC3]:0)
   [junit4] Completed on J0 in 33.92s, 1 test, 1 failure, 3 errors <<< FAILURES!

[...truncated 98 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1275: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 391 suites, 1645 tests, 3 suite-level errors, 1 error, 120 ignored (28 assumptions)

Total time: 55 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message