lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 1235 - Failure!
Date Fri, 17 Jan 2014 23:18:52 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1235/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:50597/wz_dmu/i/collection1, http://127.0.0.1:50583/wz_dmu/i/collection1, http://127.0.0.1:50630/wz_dmu/i/collection1, http://127.0.0.1:50586/wz_dmu/i/collection1, http://127.0.0.1:50577/wz_dmu/i/collection1, http://127.0.0.1:50604/wz_dmu/i/collection1, http://127.0.0.1:50590/wz_dmu/i/collection1, http://127.0.0.1:50637/wz_dmu/i/collection1, http://127.0.0.1:50611/wz_dmu/i/collection1, http://127.0.0.1:50580/wz_dmu/i/collection1, http://127.0.0.1:50617/wz_dmu/i/collection1]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:50597/wz_dmu/i/collection1, http://127.0.0.1:50583/wz_dmu/i/collection1, http://127.0.0.1:50630/wz_dmu/i/collection1, http://127.0.0.1:50586/wz_dmu/i/collection1, http://127.0.0.1:50577/wz_dmu/i/collection1, http://127.0.0.1:50604/wz_dmu/i/collection1, http://127.0.0.1:50590/wz_dmu/i/collection1, http://127.0.0.1:50637/wz_dmu/i/collection1, http://127.0.0.1:50611/wz_dmu/i/collection1, http://127.0.0.1:50580/wz_dmu/i/collection1, http://127.0.0.1:50617/wz_dmu/i/collection1]
	at __randomizedtesting.SeedInfo.seed([83AEB28E939A942D:2483C96E4C5F411]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:352)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1162)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:149)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:50597/wz_dmu/i/collection1, http://127.0.0.1:50630/wz_dmu/i/collection1, http://127.0.0.1:50586/wz_dmu/i/collection1, http://127.0.0.1:50577/wz_dmu/i/collection1, http://127.0.0.1:50604/wz_dmu/i/collection1, http://127.0.0.1:50637/wz_dmu/i/collection1, http://127.0.0.1:50611/wz_dmu/i/collection1, http://127.0.0.1:50580/wz_dmu/i/collection1, http://127.0.0.1:50617/wz_dmu/i/collection1]
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:495)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:283)
	... 46 more


REGRESSION:  org.apache.solr.cloud.OverseerRolesTest.testDistribSearch

Error Message:
could not set the new overseer

Stack Trace:
java.lang.AssertionError: could not set the new overseer
	at __randomizedtesting.SeedInfo.seed([83AEB28E939A942D:2483C96E4C5F411]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.OverseerRolesTest.addOverseerRole2ExistingNodes(OverseerRolesTest.java:120)
	at org.apache.solr.cloud.OverseerRolesTest.doTest(OverseerRolesTest.java:86)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 10328 lines...]
   [junit4] Suite: org.apache.solr.cloud.OverseerRolesTest
   [junit4]   2> 796093 T2698 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 796099 T2698 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-OverseerRolesTest-1389998362094
   [junit4]   2> 796100 T2698 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 796101 T2699 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 796203 T2698 oasc.ZkTestServer.run start zk server on port:49857
   [junit4]   2> 796204 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 796211 T2705 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c85a494 name:ZooKeeperConnection Watcher:127.0.0.1:49857 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 796211 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 796212 T2698 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 796223 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 796227 T2707 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77e5a05f name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 796228 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 796228 T2698 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 796236 T2698 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 796243 T2698 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 796248 T2698 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 796253 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 796254 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 796265 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 796265 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 796272 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 796272 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 796280 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 796280 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 796288 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 796288 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 796297 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 796298 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 796305 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 796306 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 796315 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 796315 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 796322 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 796323 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 796330 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 796331 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 796337 T2698 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 796338 T2698 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 796683 T2698 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 796692 T2698 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49860
   [junit4]   2> 796692 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 796693 T2698 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 796693 T2698 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-controljetty-1389998362341
   [junit4]   2> 796693 T2698 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-controljetty-1389998362341/'
   [junit4]   2> 796724 T2698 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1389998362341/solr.xml
   [junit4]   2> 796797 T2698 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 796797 T2698 oasc.CoreContainer.<init> New CoreContainer 424652415
   [junit4]   2> 796798 T2698 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-controljetty-1389998362341/]
   [junit4]   2> 796799 T2698 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 796800 T2698 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 796800 T2698 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 796800 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 796801 T2698 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 796801 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 796801 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 796801 T2698 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 796802 T2698 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 796811 T2698 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 796812 T2698 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 796812 T2698 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 796812 T2698 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49857/solr
   [junit4]   2> 796814 T2698 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 796815 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 796824 T2718 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e4920d4 name:ZooKeeperConnection Watcher:127.0.0.1:49857 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 796825 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 796832 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 796836 T2720 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ea29c18 name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 796836 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 796840 T2698 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 796852 T2698 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 796862 T2698 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 796882 T2698 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49860_
   [junit4]   2> 796884 T2698 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49860_
   [junit4]   2> 796892 T2698 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 796899 T2698 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 796910 T2698 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 796916 T2698 oasc.Overseer.start Overseer (id=91094932658520067-127.0.0.1:49860_-n_0000000000) starting
   [junit4]   2> 796924 T2698 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 796935 T2722 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 796935 T2698 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 796936 T2722 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 796942 T2698 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 796947 T2698 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 796956 T2721 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 796964 T2698 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1389998362341
   [junit4]   2> 797063 T2698 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1389998362341/conf/
   [junit4]   2> 797068 T2698 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 797069 T2698 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 797069 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 797071 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 797075 T2725 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77acee34 name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 797076 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 797079 T2698 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 797085 T2698 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 797421 T2698 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 797426 T2698 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49864
   [junit4]   2> 797429 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 797429 T2698 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 797429 T2698 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty1-1389998363079
   [junit4]   2> 797430 T2698 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty1-1389998363079/'
   [junit4]   2> 797460 T2698 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1389998363079/solr.xml
   [junit4]   2> 797539 T2698 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 797540 T2698 oasc.CoreContainer.<init> New CoreContainer 1848321285
   [junit4]   2> 797541 T2698 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty1-1389998363079/]
   [junit4]   2> 797543 T2698 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 797543 T2698 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 797544 T2698 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 797544 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 797544 T2698 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 797544 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 797545 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 797545 T2698 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 797545 T2698 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 797556 T2698 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 797556 T2698 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 797557 T2698 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 797557 T2698 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49857/solr
   [junit4]   2> 797558 T2698 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 797560 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 797565 T2736 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@528ba3c4 name:ZooKeeperConnection Watcher:127.0.0.1:49857 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 797565 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 797569 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 797573 T2738 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5dabe6c4 name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 797574 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 797586 T2698 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 798594 T2698 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49864_
   [junit4]   2> 798598 T2698 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49864_
   [junit4]   2> 798604 T2725 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 798605 T2738 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 798606 T2720 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 798616 T2698 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1389998363079
   [junit4]   2> 798718 T2698 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1389998363079/conf/
   [junit4]   2> 798723 T2698 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 798724 T2698 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 798724 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 799083 T2698 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 799090 T2698 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49867
   [junit4]   2> 799093 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 799093 T2698 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 799094 T2698 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1389998364719
   [junit4]   2> 799094 T2698 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1389998364719/'
   [junit4]   2> 799125 T2698 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1389998364719/solr.xml
   [junit4]   2> 799210 T2698 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 799211 T2698 oasc.CoreContainer.<init> New CoreContainer 108576873
   [junit4]   2> 799211 T2698 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1389998364719/]
   [junit4]   2> 799212 T2698 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 799213 T2698 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 799213 T2698 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 799213 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 799214 T2698 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 799214 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 799214 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 799215 T2698 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 799215 T2698 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 799224 T2698 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 799224 T2698 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 799224 T2698 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 799225 T2698 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49857/solr
   [junit4]   2> 799226 T2698 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 799227 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 799231 T2750 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61a030d0 name:ZooKeeperConnection Watcher:127.0.0.1:49857 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 799232 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 799237 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 799240 T2752 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4afe0f62 name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 799240 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 799252 T2698 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 800264 T2698 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49867_
   [junit4]   2> 800271 T2698 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49867_
   [junit4]   2> 800284 T2752 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 800285 T2738 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 800285 T2725 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 800285 T2720 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 800311 T2698 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1389998364719
   [junit4]   2> 800452 T2698 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1389998364719/conf/
   [junit4]   2> 800459 T2698 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 800461 T2698 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 800461 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 800781 T2698 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 800786 T2698 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49870
   [junit4]   2> 800788 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 800789 T2698 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 800789 T2698 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455
   [junit4]   2> 800790 T2698 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455/'
   [junit4]   2> 800822 T2698 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455/solr.xml
   [junit4]   2> 800896 T2698 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 800896 T2698 oasc.CoreContainer.<init> New CoreContainer 575997272
   [junit4]   2> 800896 T2698 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455/]
   [junit4]   2> 800898 T2698 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 800898 T2698 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 800898 T2698 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 800899 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 800899 T2698 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 800899 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 800899 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 800900 T2698 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 800900 T2698 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 800910 T2698 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 800910 T2698 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 800910 T2698 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 800911 T2698 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49857/solr
   [junit4]   2> 800911 T2698 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 800913 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 800917 T2764 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7bd1e603 name:ZooKeeperConnection Watcher:127.0.0.1:49857 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 800918 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 800924 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 800927 T2766 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5215e7b4 name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 800928 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 800939 T2698 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 801950 T2698 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49870_
   [junit4]   2> 801952 T2698 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49870_
   [junit4]   2> 801959 T2720 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 801959 T2725 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 801959 T2752 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 801960 T2738 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 801960 T2766 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 801975 T2698 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455
   [junit4]   2> 802118 T2698 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455/conf/
   [junit4]   2> 802125 T2698 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 802126 T2698 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 802127 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 802506 T2698 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 802514 T2698 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49873
   [junit4]   2> 802516 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 802518 T2698 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 802518 T2698 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty4-1389998368121
   [junit4]   2> 802519 T2698 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty4-1389998368121/'
   [junit4]   2> 802576 T2698 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1389998368121/solr.xml
   [junit4]   2> 802647 T2698 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 802648 T2698 oasc.CoreContainer.<init> New CoreContainer 487767418
   [junit4]   2> 802648 T2698 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty4-1389998368121/]
   [junit4]   2> 802650 T2698 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 802650 T2698 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 802651 T2698 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 802651 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 802652 T2698 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 802652 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 802652 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 802653 T2698 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 802653 T2698 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 802662 T2698 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 802662 T2698 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 802662 T2698 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 802663 T2698 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49857/solr
   [junit4]   2> 802663 T2698 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 802665 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 802669 T2778 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3955a37f name:ZooKeeperConnection Watcher:127.0.0.1:49857 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 802669 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 802677 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 802681 T2780 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@773ed08b name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 802681 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 802692 T2698 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 803701 T2698 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49873_
   [junit4]   2> 803704 T2698 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49873_
   [junit4]   2> 803711 T2780 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 803711 T2766 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 803711 T2720 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 803711 T2752 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 803711 T2725 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 803711 T2738 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 803722 T2698 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1389998368121
   [junit4]   2> 803821 T2698 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1389998368121/conf/
   [junit4]   2> 803827 T2698 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 803828 T2698 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 803828 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 804154 T2698 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 804158 T2698 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49876
   [junit4]   2> 804160 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 804161 T2698 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 804161 T2698 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1389998369822
   [junit4]   2> 804161 T2698 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1389998369822/'
   [junit4]   2> 804192 T2698 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1389998369822/solr.xml
   [junit4]   2> 804282 T2698 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 804282 T2698 oasc.CoreContainer.<init> New CoreContainer 308337545
   [junit4]   2> 804283 T2698 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1389998369822/]
   [junit4]   2> 804284 T2698 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 804285 T2698 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 804285 T2698 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 804285 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 804285 T2698 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 804285 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 804286 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 804286 T2698 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 804287 T2698 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 804296 T2698 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 804296 T2698 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 804297 T2698 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 804297 T2698 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49857/solr
   [junit4]   2> 804297 T2698 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 804299 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 804302 T2792 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14ed08cc name:ZooKeeperConnection Watcher:127.0.0.1:49857 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 804303 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 804308 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 804312 T2794 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d2e9a05 name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 804312 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 804322 T2698 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 805332 T2698 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49876_
   [junit4]   2> 805335 T2698 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49876_
   [junit4]   2> 805344 T2794 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 805345 T2720 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 805345 T2738 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 805345 T2766 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 805345 T2752 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 805346 T2780 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 805345 T2725 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 805358 T2698 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1389998369822
   [junit4]   2> 805501 T2698 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1389998369822/conf/
   [junit4]   2> 805509 T2698 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 805510 T2698 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 805510 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 805815 T2698 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 805821 T2698 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49879
   [junit4]   2> 805823 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 805824 T2698 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 805824 T2698 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1389998371505
   [junit4]   2> 805824 T2698 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1389998371505/'
   [junit4]   2> 805854 T2698 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1389998371505/solr.xml
   [junit4]   2> 805929 T2698 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 805930 T2698 oasc.CoreContainer.<init> New CoreContainer 1026101555
   [junit4]   2> 805930 T2698 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty6-1389998371505/]
   [junit4]   2> 805932 T2698 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 805932 T2698 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 805932 T2698 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 805933 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 805933 T2698 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 805933 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 805934 T2698 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 805934 T2698 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 805934 T2698 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 805945 T2698 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 805945 T2698 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 805945 T2698 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 805945 T2698 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49857/solr
   [junit4]   2> 805946 T2698 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 805947 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 805952 T2806 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53284aa3 name:ZooKeeperConnection Watcher:127.0.0.1:49857 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 805952 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 805958 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 805962 T2808 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2adca773 name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 805962 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 805972 T2698 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 806986 T2698 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49879_
   [junit4]   2> 806988 T2698 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49879_
   [junit4]   2> 806998 T2766 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 807000 T2720 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 807000 T2725 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 807000 T2738 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 807000 T2752 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 807000 T2794 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 807000 T2780 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 807000 T2808 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 807355 T2698 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1389998371505
   [junit4]   2> 807459 T2698 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1389998371505/conf/
   [junit4]   2> 807463 T2698 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 807464 T2698 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 807465 T2698 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 807466 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 807471 T2811 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ad68e17 name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 807471 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 807475 T2698 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 807487 T2698 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 807490 T2813 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74e411d8 name:ZooKeeperConnection Watcher:127.0.0.1:49857/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 807491 T2698 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 807493 T2698 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 807501 T2746 oasha.CollectionsHandler.handleCreateAction Creating Collection : maxShardsPerNode=2&action=CREATE&version=2&replicationFactor=2&numShards=4&wt=javabin&name=testOverseerCol
   [junit4]   2> 807504 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807507 T2722 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"testOverseerCol",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 807507 T2722 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"testOverseerCol",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 807509 T2722 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
   [junit4]   2> 807509 T2722 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/testOverseerCol 
   [junit4]   2> 807509 T2722 oascc.SolrZkClient.makePath makePath: /collections/testOverseerCol
   [junit4]   2> 807521 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807524 T2721 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 807526 T2721 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: testOverseerCol
   [junit4]   2> 807526 T2721 oasc.Overseer$ClusterStateUpdater.createCollection Create collection testOverseerCol with shards [shard1, shard2, shard3, shard4]
   [junit4]   2> 807533 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807536 T2808 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807536 T2811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807536 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807537 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807536 T2780 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807536 T2720 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807536 T2794 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807536 T2813 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807536 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807536 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807623 T2722 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2, shard3, shard4] , repFactor : 2
   [junit4]   2> 807623 T2722 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica1 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:49864_
   [junit4]   2> 807624 T2722 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica2 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:49879_
   [junit4]   2> 807624 T2722 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica1 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:49867_
   [junit4]   2> 807625 T2722 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica2 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:49860_
   [junit4]   2> 807626 T2722 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica1 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:49870_
   [junit4]   2> 807628 T2722 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica2 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:49876_
   [junit4]   2> 807629 T2722 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica1 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:49873_
   [junit4]   2> 807630 T2722 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica2 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:49864_
   [junit4]   2> 807641 T2771 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica1 state=down
   [junit4]   2> 807652 T2729 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica1 state=down
   [junit4]   2> 807653 T2786 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica2 state=down
   [junit4]   2> 807654 T2762 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica1 state=down
   [junit4]   2> 807663 T2803 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica2 state=down
   [junit4]   2> 807668 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807668 T2786 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 807668 T2729 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 807670 T2713 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica2 state=down
   [junit4]   2> 807670 T2762 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 807671 T2730 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica2 state=down
   [junit4]   2> 807674 T2747 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica1 state=down
   [junit4]   2> 807674 T2803 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 807676 T2771 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 807677 T2747 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 807679 T2713 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 807679 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807680 T2721 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 807681 T2721 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49873",
   [junit4]   2> 	  "core":"testOverseerCol_shard4_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49873_",
   [junit4]   2> 	  "shard":"shard4",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 807682 T2730 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 807688 T2721 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49876",
   [junit4]   2> 	  "core":"testOverseerCol_shard3_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49876_",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 807694 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807695 T2721 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49864",
   [junit4]   2> 	  "core":"testOverseerCol_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49864_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 807699 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807701 T2721 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49870",
   [junit4]   2> 	  "core":"testOverseerCol_shard3_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49870_",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 807709 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807711 T2721 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49879",
   [junit4]   2> 	  "core":"testOverseerCol_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49879_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 807717 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807720 T2721 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49860",
   [junit4]   2> 	  "core":"testOverseerCol_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49860_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 807725 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807728 T2721 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49867",
   [junit4]   2> 	  "core":"testOverseerCol_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49867_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 807734 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807736 T2721 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49864",
   [junit4]   2> 	  "core":"testOverseerCol_shard4_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49864_",
   [junit4]   2> 	  "shard":"shard4",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 807742 T2720 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 807849 T2808 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807849 T2813 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807849 T2811 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807849 T2780 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807849 T2794 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807849 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807849 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807849 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807849 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 807849 T2720 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 808670 T2729 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica1
   [junit4]   2> 808670 T2786 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica2
   [junit4]   2> 808671 T2762 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica1
   [junit4]   2> 808672 T2729 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard1_replica1' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty1-1389998363079/testOverseerCol_shard1_replica1
   [junit4]   2> 808673 T2729 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 808673 T2762 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard3_replica1' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455/testOverseerCol_shard3_replica1
   [junit4]   2> 808674 T2762 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 808675 T2786 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard3_replica2' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1389998369822/testOverseerCol_shard3_replica2
   [junit4]   2> 808675 T2786 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 808676 T2803 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica2
   [junit4]   2> 808676 T2762 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 808676 T2762 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 808677 T2771 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica1
   [junit4]   2> 808678 T2803 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard1_replica2' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1389998371505/testOverseerCol_shard1_replica2
   [junit4]   2> 808678 T2803 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 808679 T2747 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica1
   [junit4]   2> 808678 T2729 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 808680 T2729 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 808680 T2786 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 808681 T2786 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 808681 T2747 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard2_replica1' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1389998364719/testOverseerCol_shard2_replica1
   [junit4]   2> 808679 T2771 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard4_replica1' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty4-1389998368121/testOverseerCol_shard4_replica1
   [junit4]   2> 808682 T2771 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 808682 T2747 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 808683 T2803 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 808684 T2803 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 808683 T2730 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica2
   [junit4]   2> 808682 T2713 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica2
   [junit4]   2> 808687 T2730 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard4_replica2' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty1-1389998363079/testOverseerCol_shard4_replica2
   [junit4]   2> 808687 T2730 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 808687 T2713 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard2_replica2' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-controljetty-1389998362341/testOverseerCol_shard2_replica2
   [junit4]   2> 808688 T2713 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 808684 T2762 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455/testOverseerCol_shard3_replica1/'
   [junit4]   2> 808688 T2747 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 808689 T2747 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 808688 T2771 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 808690 T2771 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 808690 T2730 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 808690 T2730 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 808689 T2786 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1389998369822/testOverseerCol_shard3_replica2/'
   [junit4]   2> 808689 T2729 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty1-1389998363079/testOverseerCol_shard1_replica1/'
   [junit4]   2> 808694 T2713 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 808694 T2713 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 808703 T2803 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1389998371505/testOverseerCol_shard1_replica2/'
   [junit4]   2> 808712 T2747 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1389998364719/testOverseerCol_shard2_replica1/'
   [junit4]   2> 808716 T2713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-controljetty-1389998362341/testOverseerCol_shard2_replica2/'
   [junit4]   2> 808717 T2771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty4-1389998368121/testOverseerCol_shard4_replica1/'
   [junit4]   2> 808721 T2730 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty1-1389998363079/testOverseerCol_shard4_replica2/'
   [junit4]   2> 808972 T2729 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 808984 T2786 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 808989 T2762 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 809035 T2713 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 809038 T2747 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 809044 T2771 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 809056 T2803 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 809146 T2730 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 809231 T2762 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809239 T2762 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 809290 T2762 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica1] Schema name=test
   [junit4]   2> 809329 T2803 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809336 T2713 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809336 T2803 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 809347 T2803 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica2] Schema name=test
   [junit4]   2> 809348 T2786 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809351 T2786 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 809364 T2713 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 809364 T2786 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica2] Schema name=test
   [junit4]   2> 809392 T2713 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica2] Schema name=test
   [junit4]   2> 809447 T2730 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809454 T2730 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 809459 T2747 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809465 T2730 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica2] Schema name=test
   [junit4]   2> 809470 T2747 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 809474 T2729 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809479 T2747 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica1] Schema name=test
   [junit4]   2> 809486 T2729 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 809498 T2771 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809506 T2771 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 809514 T2729 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica1] Schema name=test
   [junit4]   2> 809515 T2771 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica1] Schema name=test
   [junit4]   2> 811809 T2762 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 811819 T2762 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 811823 T2762 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 811997 T2762 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 812046 T2762 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 812068 T2762 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 812080 T2762 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 812081 T2762 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 812081 T2762 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 812096 T2762 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 812096 T2762 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 812097 T2762 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 812097 T2762 oasc.SolrCore.<init> [testOverseerCol_shard3_replica1] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455/testOverseerCol_shard3_replica1/, dataDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455/testOverseerCol_shard3_replica1/data/
   [junit4]   2> 812098 T2762 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b92a493
   [junit4]   2> 812106 T2762 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1389998366455/testOverseerCol_shard3_replica1/data
   [junit4]   2> 812107 T2762 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../Users/jenkins/wo

[...truncated too long message...]

626 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1389999100622/jetty12
   [junit4]   2> 1626660 T4626 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/wz_dmu/i,null}
   [junit4]   2> 1627312 T4626 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 1627320 T4626 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:50570 50570
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   1> safe stop:
   [junit4]   1> safe stop:
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.method=testDistribSearch -Dtests.seed=83AEB28E939A942D -Dtests.slow=true -Dtests.locale=lt_LT -Dtests.timezone=Asia/Choibalsan -Dtests.file.encoding=UTF-8
   [junit4] ERROR   92.7s | ChaosMonkeySafeLeaderTest.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:50597/wz_dmu/i/collection1, http://127.0.0.1:50583/wz_dmu/i/collection1, http://127.0.0.1:50630/wz_dmu/i/collection1, http://127.0.0.1:50586/wz_dmu/i/collection1, http://127.0.0.1:50577/wz_dmu/i/collection1, http://127.0.0.1:50604/wz_dmu/i/collection1, http://127.0.0.1:50590/wz_dmu/i/collection1, http://127.0.0.1:50637/wz_dmu/i/collection1, http://127.0.0.1:50611/wz_dmu/i/collection1, http://127.0.0.1:50580/wz_dmu/i/collection1, http://127.0.0.1:50617/wz_dmu/i/collection1]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([83AEB28E939A942D:2483C96E4C5F411]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:352)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:640)
   [junit4]    > 	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1162)
   [junit4]    > 	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:149)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:50597/wz_dmu/i/collection1, http://127.0.0.1:50630/wz_dmu/i/collection1, http://127.0.0.1:50586/wz_dmu/i/collection1, http://127.0.0.1:50577/wz_dmu/i/collection1, http://127.0.0.1:50604/wz_dmu/i/collection1, http://127.0.0.1:50637/wz_dmu/i/collection1, http://127.0.0.1:50611/wz_dmu/i/collection1, http://127.0.0.1:50580/wz_dmu/i/collection1, http://127.0.0.1:50617/wz_dmu/i/collection1]
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:495)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:283)
   [junit4]    > 	... 46 more
   [junit4]   2> 1627361 T4626 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 92745 T4625 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> 1628144 T4959 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1628144 T4959 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1628145 T4959 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=lt_LT, timezone=Asia/Choibalsan
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.7.0_51 (64-bit)/cpus=2,threads=5,free=218020304,total=482324480
   [junit4]   2> NOTE: All tests run in this JVM: [LeaderElectionTest, CopyFieldTest, HdfsUnloadDistributedZkTest, SuggesterWFSTTest, TestCollationFieldDocValues, DisMaxRequestHandlerTest, TestLMJelinekMercerSimilarityFactory, TestComponentsName, TestDefaultSimilarityFactory, RAMDirectoryFactoryTest, TestSolr4Spatial, EchoParamsTest, TestDocumentBuilder, TestFieldResource, TestClassNameShortening, SpatialFilterTest, TestLMDirichletSimilarityFactory, ModifyConfFileTest, TestFoldingMultitermQuery, LukeRequestHandlerTest, DistributedSuggestComponentTest, TestFuzzyAnalyzedSuggestions, TestLuceneMatchVersion, MBeansHandlerTest, MultiTermTest, CSVRequestHandlerTest, RangeFacetTest, RequiredFieldsTest, OpenExchangeRatesOrgProviderTest, RegexBoostProcessorTest, TestDocBasedVersionConstraints, CoreAdminHandlerTest, FullSolrCloudDistribCmdsTest, SortByFunctionTest, AssignTest, TestFunctionQuery, SolrCoreCheckLockOnStartupTest, TestRandomDVFaceting, TestSolrIndexConfig, SolrIndexConfigTest, SchemaVersionSpecificBehaviorTest, NoFacetTest, TestCharFilters, TestExtendedDismaxParser, QueryElevationComponentTest, TestPseudoReturnFields, BasicZkTest, TestSolrXmlPersistor, TestFieldTypeCollectionResource, TestFieldTypeResource, OpenCloseCoreStressTest, FileBasedSpellCheckerTest, TestNoOpRegenerator, HdfsRecoveryZkTest, AnalysisAfterCoreReloadTest, PrimitiveFieldTypeTest, TestCoreDiscovery, PrimUtilsTest, CollectionsAPIDistributedZkTest, LeaderElectionIntegrationTest, TestQuerySenderNoQuery, IndexBasedSpellCheckerTest, WordBreakSolrSpellCheckerTest, HdfsDirectoryTest, SliceStateTest, SpellCheckComponentTest, OverseerTest, HighlighterTest, FileUtilsTest, CachingDirectoryFactoryTest, TestPostingsSolrHighlighter, TestBM25SimilarityFactory, TestStressReorder, TestSuggestSpellingConverter, IndexSchemaRuntimeFieldTest, TestSchemaNameResource, TestRandomMergePolicy, TestUniqueKeyFieldResource, DocumentBuilderTest, TestSurroundQueryParser, HdfsSyncSliceTest, TestSearchPerf, TestDistributedMissingSort, AlternateDirectoryTest, TestPHPSerializedResponseWriter, LoggingHandlerTest, TestXIncludeConfig, TestValueSourceCache, CircularListTest, TestPartialUpdateDeduplication, HdfsChaosMonkeySafeLeaderTest, PeerSyncTest, ResourceLoaderTest, QueryParsingTest, DefaultValueUpdateProcessorTest, TestIndexingPerformance, AddSchemaFieldsUpdateProcessorFactoryTest, NumericFieldsTest, TestSolrDeletionPolicy1, HighlighterConfigTest, FunctionTest, TestMultiCoreConfBootstrap, AddBlockUpdateTest, DistanceFunctionTest, TestIndexSearcher, TestCursorMarkWithoutUniqueKey, TestAnalyzedSuggestions, TestCustomSort, ChangedSchemaMergeTest, JSONWriterTest, HdfsCollectionsAPIDistributedZkTest, TestDFRSimilarityFactory, DocValuesMissingTest, AutoCommitTest, TestElisionMultitermQuery, InfoHandlerTest, TestZkChroot, SampleTest, SynonymTokenizerTest, AbstractAnalyticsStatsTest, BlockDirectoryTest, TestJmxMonitoredMap, TestPhraseSuggestions, TestSolrQueryParserResource, ResponseLogComponentTest, StatsComponentTest, DeleteReplicaTest, OverseerRolesTest, DirectSolrConnectionTest, TestFastOutputStream, BJQParserTest, SearchHandlerTest, TestWordDelimiterFilterFactory, TestCopyFieldCollectionResource, ConvertedLegacyTest, TestMaxScoreQueryParser, ScriptEngineTest, TestSchemaVersionResource, ChaosMonkeyNothingIsSafeTest, TestQueryTypes, TestHighFrequencyDictionaryFactory, TestJmxIntegration, IndexSchemaTest, TestJoin, TestHashPartitioner, AliasIntegrationTest, SolrCmdDistributorTest, DistributedQueryComponentCustomSortTest, TestRecovery, TestImplicitCoreProperties, TestRemoteStreaming, LegacyHTMLStripCharFilterTest, TestCloudManagedSchemaAddField, TestQueryUtils, HdfsLockFactoryTest, TestSolrQueryParserDefaultOperatorResource, ConnectionManagerTest, StatelessScriptUpdateProcessorFactoryTest, TestLRUCache, TestFastWriter, TestRealTimeGet, TestShardHandlerFactory, CoreMergeIndexesAdminHandlerTest, TestReloadAndDeleteDocs, TestSchemaResource, HardAutoCommitTest, TestFileDictionaryLookup, TestWriterPerf, BlockCacheTest, SyncSliceTest, TestStressRecovery, TestCollationField, TestIBSimilarityFactory, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest]
   [junit4] Completed in 93.81s, 1 test, 1 error <<< FAILURES!

[...truncated 556 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:453: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:433: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:37: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:189: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1304: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:937: There were test failures: 364 suites, 1596 tests, 1 error, 1 failure, 39 ignored (5 assumptions)

Total time: 85 minutes 19 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message