lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.7.0_60-ea-b04) - Build # 9479 - Still Failing!
Date Tue, 25 Feb 2014 01:24:40 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9479/
Java: 64bit/jdk1.7.0_60-ea-b04 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
Could not find collection:addReplicaColl

Stack Trace:
java.lang.AssertionError: Could not find collection:addReplicaColl
	at __randomizedtesting.SeedInfo.seed([A37F55D1904407A2:2299DBC9E71B679E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:147)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:724)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.addReplicaTest(CollectionsAPIDistributedZkTest.java:1189)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:209)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 10661 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> 737628 T3567 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 737629 T3567 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 737632 T3567 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-CollectionsAPIDistributedZkTest-1393290728104
   [junit4]   2> 737632 T3567 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 737633 T3568 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 737733 T3567 oasc.ZkTestServer.run start zk server on port:56541
   [junit4]   2> 737734 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 737815 T3574 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@358d16b5 name:ZooKeeperConnection Watcher:127.0.0.1:56541 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 737816 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 737816 T3567 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 737825 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 737826 T3576 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5450e42c name:ZooKeeperConnection Watcher:127.0.0.1:56541/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 737827 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 737827 T3567 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 737833 T3567 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 737838 T3567 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 737842 T3567 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 737847 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 737847 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 737855 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 737856 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 737862 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 737862 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 737868 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 737869 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 737873 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 737874 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 737880 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 737881 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 737886 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 737887 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 737891 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 737892 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 737897 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 737898 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 737903 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 737903 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 737908 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 737909 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 737916 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 737918 T3578 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f9f73e4 name:ZooKeeperConnection Watcher:127.0.0.1:56541/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 737918 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> Using new style solr.xml
   [junit4]   2> 737922 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 737924 T3580 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@164d2fc2 name:ZooKeeperConnection Watcher:127.0.0.1:56541 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 737925 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 737925 T3567 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 737931 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 737933 T3582 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4516bb65 name:ZooKeeperConnection Watcher:127.0.0.1:56541/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 737934 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 737935 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 737935 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 737943 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 737943 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
   [junit4]   2> 737950 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 737951 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 737956 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 737957 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 737962 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 737962 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
   [junit4]   2> 737968 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 737969 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
   [junit4]   2> 737973 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 737974 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 737978 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 737979 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 737984 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 737985 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 737990 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 737990 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 737996 T3567 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 737996 T3567 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
   [junit4]   2> 738190 T3567 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 738193 T3567 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 738196 T3567 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:43237
   [junit4]   2> 738197 T3567 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 738197 T3567 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 738197 T3567 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474
   [junit4]   2> 738197 T3567 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/'
   [junit4]   2> 738206 T3567 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/solr.xml
   [junit4]   2> 738227 T3567 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474
   [junit4]   2> 738227 T3567 oasc.CoreContainer.<init> New CoreContainer 1530904107
   [junit4]   2> 738227 T3567 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/]
   [junit4]   2> 738228 T3567 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 738228 T3567 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 738228 T3567 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 738229 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 738229 T3567 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 738229 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 738229 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 738229 T3567 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 738229 T3567 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 738233 T3567 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 738233 T3567 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 738233 T3567 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 738233 T3567 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56541/solr
   [junit4]   2> 738234 T3567 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 738234 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 738235 T3593 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a279d15 name:ZooKeeperConnection Watcher:127.0.0.1:56541 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 738236 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 738238 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 738239 T3595 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@540b6dad name:ZooKeeperConnection Watcher:127.0.0.1:56541/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 738239 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 738240 T3567 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 738243 T3567 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 738247 T3567 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 738248 T3567 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43237_
   [junit4]   2> 738249 T3567 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43237_
   [junit4]   2> 738252 T3567 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 738253 T3567 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 738257 T3567 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:43237_
   [junit4]   2> 738257 T3567 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 738259 T3567 oasc.Overseer.start Overseer (id=91310701157220358-127.0.0.1:43237_-n_0000000000) starting
   [junit4]   2> 738262 T3567 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 738265 T3597 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 738266 T3567 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 738266 T3597 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 738268 T3567 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 738269 T3567 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 738271 T3596 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 738273 T3567 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474
   [junit4]   2> 738282 T3567 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/conf/
   [junit4]   2> 738354 T3567 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 738354 T3567 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 738354 T3567 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 738357 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 738360 T3600 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42acbb1e name:ZooKeeperConnection Watcher:127.0.0.1:56541/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 738360 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 738362 T3567 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 738364 T3567 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 738540 T3567 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 738541 T3567 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 738546 T3567 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 738547 T3567 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:47216
   [junit4]   2> 738549 T3567 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 738549 T3567 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 738550 T3567 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837
   [junit4]   2> 738550 T3567 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/'
   [junit4]   2> 738578 T3567 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/solr.xml
   [junit4]   2> 738605 T3567 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837
   [junit4]   2> 738605 T3567 oasc.CoreContainer.<init> New CoreContainer 1318066173
   [junit4]   2> 738605 T3567 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/]
   [junit4]   2> 738606 T3567 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 738606 T3567 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 738606 T3567 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 738606 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 738607 T3567 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 738607 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 738607 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 738607 T3567 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 738607 T3567 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 738610 T3567 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 738611 T3567 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 738611 T3567 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 738611 T3567 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56541/solr
   [junit4]   2> 738611 T3567 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 738612 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 738613 T3611 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b9a1a03 name:ZooKeeperConnection Watcher:127.0.0.1:56541 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 738614 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 738618 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 738619 T3613 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@352deda3 name:ZooKeeperConnection Watcher:127.0.0.1:56541/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 738620 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 738628 T3567 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 739633 T3567 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47216_
   [junit4]   2> 739634 T3567 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47216_
   [junit4]   2> 739636 T3595 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 739636 T3613 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 739636 T3600 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 739640 T3567 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837
   [junit4]   2> 739644 T3567 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/conf/
   [junit4]   2> 739713 T3567 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 739713 T3567 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 739713 T3567 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 739893 T3567 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 739894 T3567 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 739897 T3567 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 739897 T3567 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:56542
   [junit4]   2> 739899 T3567 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 739899 T3567 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 739899 T3567 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187
   [junit4]   2> 739900 T3567 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/'
   [junit4]   2> 739908 T3567 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/solr.xml
   [junit4]   2> 739927 T3567 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187
   [junit4]   2> 739927 T3567 oasc.CoreContainer.<init> New CoreContainer 482718206
   [junit4]   2> 739927 T3567 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/]
   [junit4]   2> 739928 T3567 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 739928 T3567 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 739928 T3567 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 739928 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 739928 T3567 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 739929 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 739929 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 739929 T3567 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 739929 T3567 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 739932 T3567 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 739932 T3567 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 739933 T3567 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 739933 T3567 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56541/solr
   [junit4]   2> 739933 T3567 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 739934 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 739936 T3625 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7932879 name:ZooKeeperConnection Watcher:127.0.0.1:56541 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 739937 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 739941 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 739942 T3627 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15f21207 name:ZooKeeperConnection Watcher:127.0.0.1:56541/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 739943 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 739951 T3567 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 740954 T3567 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56542_
   [junit4]   2> 740957 T3567 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56542_
   [junit4]   2> 740961 T3595 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 740961 T3613 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 740961 T3600 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 740962 T3627 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 740972 T3567 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187
   [junit4]   2> 740982 T3567 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/conf/
   [junit4]   2> 741120 T3567 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 741122 T3567 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 741123 T3567 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 741455 T3567 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 741456 T3567 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 741484 T3567 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 741485 T3567 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:40160
   [junit4]   2> 741488 T3567 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 741488 T3567 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 741489 T3567 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1393290731597
   [junit4]   2> 741489 T3567 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1393290731597/'
   [junit4]   2> 741507 T3567 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1393290731597/solr.xml
   [junit4]   2> 741562 T3567 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1393290731597
   [junit4]   2> 741562 T3567 oasc.CoreContainer.<init> New CoreContainer 1730340543
   [junit4]   2> 741563 T3567 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1393290731597/]
   [junit4]   2> 741564 T3567 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 741564 T3567 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 741565 T3567 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 741565 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 741565 T3567 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 741566 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 741566 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 741566 T3567 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 741567 T3567 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 741572 T3567 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 741573 T3567 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 741573 T3567 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 741574 T3567 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56541/solr
   [junit4]   2> 741574 T3567 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 741575 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 741576 T3639 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@702d4fcc name:ZooKeeperConnection Watcher:127.0.0.1:56541 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 741576 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 741581 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 741595 T3641 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4059e772 name:ZooKeeperConnection Watcher:127.0.0.1:56541/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 741599 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 741620 T3567 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 742624 T3567 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40160_
   [junit4]   2> 742625 T3567 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40160_
   [junit4]   2> 742637 T3627 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 742637 T3641 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 742637 T3595 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 742637 T3613 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 742637 T3600 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 742648 T3567 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1393290731597
   [junit4]   2> 742659 T3567 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1393290731597/conf/
   [junit4]   2> 742747 T3567 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 742748 T3567 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 742748 T3567 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 742943 T3567 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 742943 T3567 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 742947 T3567 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 742948 T3567 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:38539
   [junit4]   2> 742949 T3567 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 742949 T3567 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 742950 T3567 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1393290733223
   [junit4]   2> 742950 T3567 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1393290733223/'
   [junit4]   2> 742959 T3567 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1393290733223/solr.xml
   [junit4]   2> 742978 T3567 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1393290733223
   [junit4]   2> 742979 T3567 oasc.CoreContainer.<init> New CoreContainer 610244272
   [junit4]   2> 742979 T3567 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1393290733223/]
   [junit4]   2> 742980 T3567 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 742980 T3567 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 742980 T3567 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 742980 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 742980 T3567 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 742980 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 742981 T3567 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 742981 T3567 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 742981 T3567 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 742984 T3567 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 742984 T3567 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 742985 T3567 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 742985 T3567 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56541/solr
   [junit4]   2> 742985 T3567 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 742986 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 742988 T3653 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78533e2a name:ZooKeeperConnection Watcher:127.0.0.1:56541 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 742988 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 742992 T3567 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 742994 T3655 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35cd8338 name:ZooKeeperConnection Watcher:127.0.0.1:56541/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 742994 T3567 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 743004 T3567 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 744010 T3567 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38539_
   [junit4]   2> 744012 T3567 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38539_
   [junit4]   2> 744018 T3600 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 744018 T3613 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 744019 T3595 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 744019 T3627 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 744019 T3655 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 744019 T3641 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 744028 T3567 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1393290733223
   [junit4]   2> 744037 T3567 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1393290733223/conf/
   [junit4]   2> 744105 T3567 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 744106 T3567 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 744106 T3567 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 744181 T3606 oasha.CollectionsHandler.handleCreateAction Creating Collection : name=solrj_collection&action=CREATE&version=2&router.field=myOwnField&collection.configName=conf1&wt=javabin&numShards=2&router.name=compositeId&replicationFactor=2
   [junit4]   2> 744185 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 744190 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 744190 T3597 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField",
   [junit4]   2> 	  "router.name":"compositeId"}
   [junit4]   2> 744191 T3597 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField",
   [junit4]   2> 	  "router.name":"compositeId"}
   [junit4]   2> 744193 T3597 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection 
   [junit4]   2> 744193 T3597 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
   [junit4]   2> 744201 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 744204 T3596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 744206 T3596 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
   [junit4]   2> 744207 T3596 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
   [junit4]   2> 744211 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 744212 T3627 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744212 T3595 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744213 T3613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744212 T3641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744212 T3600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744212 T3655 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744302 T3597 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2] , repFactor : 2
   [junit4]   2> 744302 T3597 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:56542_
   [junit4]   2> 744305 T3597 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:38539_
   [junit4]   2> 744309 T3597 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:47216_
   [junit4]   2> 744314 T3597 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:43237_
   [junit4]   2> 744373 T3589 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down
   [junit4]   2> 744375 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 744375 T3589 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 744377 T3596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 744377 T3596 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:43237",
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:43237_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 744381 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 744386 T3623 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down
   [junit4]   2> 744387 T3651 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down
   [junit4]   2> 744387 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 744387 T3623 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 744388 T3651 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 744389 T3596 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:56542",
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:56542_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 744391 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 744393 T3607 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down
   [junit4]   2> 744393 T3596 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:38539",
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:38539_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 744395 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 744395 T3607 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 744402 T3596 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:47216",
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:47216_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 744410 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 744515 T3613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744516 T3627 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744516 T3655 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744515 T3641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744515 T3595 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 744515 T3600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 745376 T3589 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
   [junit4]   2> 745377 T3589 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/solrj_collection_shard2_replica2
   [junit4]   2> 745377 T3589 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 745378 T3589 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 745378 T3589 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 745379 T3589 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 745380 T3589 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/solrj_collection_shard2_replica2/'
   [junit4]   2> 745388 T3623 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
   [junit4]   2> 745388 T3651 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
   [junit4]   2> 745389 T3623 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/solrj_collection_shard1_replica1
   [junit4]   2> 745389 T3651 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1393290733223/solrj_collection_shard1_replica2
   [junit4]   2> 745389 T3651 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 745389 T3623 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 745390 T3651 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 745390 T3623 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 745391 T3651 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 745391 T3623 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 745392 T3651 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 745392 T3651 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1393290733223/solrj_collection_shard1_replica2/'
   [junit4]   2> 745393 T3623 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 745393 T3623 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/solrj_collection_shard1_replica1/'
   [junit4]   2> 745395 T3607 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
   [junit4]   2> 745396 T3607 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/solrj_collection_shard2_replica1
   [junit4]   2> 745396 T3607 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 745397 T3607 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 745397 T3607 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 745399 T3607 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 745399 T3607 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/solrj_collection_shard2_replica1/'
   [junit4]   2> 745430 T3589 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 745449 T3607 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 745452 T3651 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 745456 T3623 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 745489 T3607 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 745491 T3607 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 745494 T3623 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 745496 T3623 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 745497 T3589 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 745499 T3589 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 745500 T3607 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
   [junit4]   2> 745515 T3623 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
   [junit4]   2> 745517 T3651 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 745519 T3589 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
   [junit4]   2> 745523 T3651 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 745534 T3651 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
   [junit4]   2> 745840 T3607 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 745840 T3607 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 745841 T3607 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 745849 T3607 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 745853 T3607 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 745865 T3589 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 745866 T3589 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 745867 T3589 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 745868 T3607 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 745877 T3607 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 745877 T3589 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 745880 T3589 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 745881 T3607 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 745882 T3607 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 745882 T3607 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 745883 T3607 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 745884 T3607 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 745884 T3607 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 745885 T3607 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 745885 T3607 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/solrj_collection_shard2_replica1/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/solrj_collection_shard2_replica1/data/
   [junit4]   2> 745886 T3607 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cf089eb
   [junit4]   2> 745887 T3607 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/solrj_collection_shard2_replica1/data
   [junit4]   2> 745887 T3607 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/solrj_collection_shard2_replica1/data/index/
   [junit4]   2> 745887 T3623 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 745888 T3623 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 745887 T3607 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 745888 T3623 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 745889 T3607 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1393290728837/solrj_collection_shard2_replica1/data/index
   [junit4]   2> 745889 T3607 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=85.2666015625, floorSegmentMB=1.5732421875, forceMergeDeletesPctAllowed=7.356801650969239, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6494859534224845
   [junit4]   2> 745890 T3607 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3a8489ba lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@70f1678),segFN=segments_1,generation=1}
   [junit4]   2> 745890 T3607 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 745893 T3607 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 745894 T3607 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 745894 T3607 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 745894 T3607 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 745894 T3607 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 745895 T3607 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 745895 T3623 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 745895 T3607 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 745895 T3607 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 745895 T3607 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 745896 T3607 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 745896 T3607 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 745897 T3607 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 745897 T3623 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 745897 T3607 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 745897 T3607 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 745898 T3607 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 745898 T3589 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 745898 T3607 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 745901 T3607 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 745901 T3589 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 745903 T3607 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 745903 T3607 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 745903 T3607 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1253698889, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 745904 T3607 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3a8489ba lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@70f1678),segFN=segments_1,generation=1}
   [junit4]   2> 745904 T3607 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 745904 T3607 oass.SolrIndexSearcher.<init> Opening Searcher@54af9ce0[solrj_collection_shard2_replica1] main
   [junit4]   2> 745905 T3589 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 745906 T3589 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 745906 T3589 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 745906 T3589 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 745907 T3661 oasc.SolrCore.registerSearcher [solrj_collection_shard2_replica1] Registered new searcher Searcher@54af9ce0[solrj_collection_shard2_replica1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 745907 T3623 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 745907 T3589 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 745907 T3607 oasc.CoreContainer.registerCore registering core: solrj_collection_shard2_replica1
   [junit4]   2> 745908 T3589 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 745908 T3607 oasc.ZkController.register Register replica - core:solrj_collection_shard2_replica1 address:https://127.0.0.1:47216 collection:solrj_collection shard:shard2
   [junit4]   2> 745908 T3589 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 745909 T3589 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/solrj_collection_shard2_replica2/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/solrj_collection_shard2_replica2/data/
   [junit4]   2> 745909 T3589 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cf089eb
   [junit4]   2> 745909 T3607 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard2/election
   [junit4]   2> 745910 T3589 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/solrj_collection_shard2_replica2/data
   [junit4]   2> 745910 T3589 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/solrj_collection_shard2_replica2/data/index/
   [junit4]   2> 745910 T3623 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 745910 T3589 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 745911 T3589 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1393290728474/solrj_collection_shard2_replica2/data/index
   [junit4]   2> 745911 T3589 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=85.2666015625, floorSegmentMB=1.5732421875, forceMergeDeletesPctAllowed=7.356801650969239, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6494859534224845
   [junit4]   2> 745912 T3589 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f90554e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@295b37ad),segFN=segments_1,generation=1}
   [junit4]   2> 745912 T3589 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 745914 T3623 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 745915 T3607 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 745915 T3623 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 745916 T3623 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 745916 T3623 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 745916 T3589 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 745916 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 745916 T3589 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 745916 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 745917 T3589 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 745917 T3623 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 745917 T3623 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 745917 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 745917 T3607 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=179999
   [junit4]   2> 745918 T3596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 745918 T3623 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 745917 T3589 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 745918 T3623 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/solrj_collection_shard1_replica1/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/solrj_collection_shard1_replica1/data/
   [junit4]   2> 745918 T3589 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 745919 T3623 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cf089eb
   [junit4]   2> 745919 T3589 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 745919 T3589 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 745920 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 745920 T3623 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/solrj_collection_shard1_replica1/data
   [junit4]   2> 745920 T3589 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 745920 T3623 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/solrj_collection_shard1_replica1/data/index/
   [junit4]   2> 745920 T3589 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 745921 T3623 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 745921 T3589 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 745921 T3623 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1393290730187/solrj_collection_shard1_replica1/data/index
   [junit4]   2> 745922 T3589 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 745922 T3589 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 745922 T3623 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=85.2666015625, floorSegmentMB=1.5732421875, forceMergeDeletesPctAllowed=7.356801650969239, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6494859534224845
   [junit4]   2> 745922 T3589 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 745923 T3623 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@19ca3f0f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@47643483),segFN=segments_1,generation=1}
   [junit4]   2> 745923 T3623 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 745923 T3589 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 745924 T3589 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 745924 T3589 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 745926 T3589 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 745926 T3623 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 745927 T3623 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 745927 T3623 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 745927 T3623 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 745927 T3623 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 745927 T3623 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 745928 T3589 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 745928 T3623 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 745928 T3589 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 745928 T3623 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 745928 T3623 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 745929 T3589 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1253698889, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 745929 T3623 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 745930 T3589 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f90554e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@295b37ad),segFN=segments_1,generation=1}
   [junit4]   2> 745930 T3623 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 745930 T3623 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 745930 T3589 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 745930 T3623 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 745931 T3589 oass.SolrIndexSearcher.<init> Opening Searcher@4b9758d9[solrj_collection_shard2_replica2] main
   [junit4]   2> 745931 T3623 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 745931 T3623 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 745932 T3623 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 745933 T3663 oasc.SolrCore.registerSearcher [solrj_collection_shard2_replica2] Registered new searcher Searcher@4b9758d9[solrj_collection_shard2_replica2] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 745934 T3623 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 745935 T3589 oasc.CoreContainer.registerCore registering core: solrj_collection_shard2_replica2
   [junit4]   2> 745935 T3589 oasc.ZkController.register Register replica - core:solrj_collection_shard2_replica2 address:https://127.0.0.1:43237 collection:solrj_collection shard:shard2
   [junit4]   2> 745936 T3623 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 745936 T3623 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 745937 T3623 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1253698889, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 745937 T3623 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@19ca3f0f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@47643483),segFN=segments_1,generation=1}
   [junit4]   2> 745938 T3623 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 745938 T3623 oass.SolrIndexSearcher.<init> Opening Searcher@b6a84f6[solrj_collection_shard1_replica1] main
   [junit4]   2> 745940 T3665 oasc.SolrCore.registerSearcher [solrj_collection_shard1_replica1] Registered new searcher Searcher@b6a84f6[solrj_collection_shard1_replica1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 745940 T3623 oasc.CoreContainer.registerCore registering core: solrj_collection_shard1_replica1
   [junit4]   2> 745941 T3623 oasc.ZkController.register Register replica - core:solrj_collection_shard1_replica1 address:https://127.0.0.1:56542 collection:solrj_collection shard:shard1
   [junit4]   2> 745942 T3623 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard1/election
   [junit4]   2> 745948 T3623 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 745950 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 745951 T3651 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 745951 T3623 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=179999
   [junit4]   2> 745951 T3651 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 745952 T3651 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 745953 T3595 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 745959 T3651 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 745962 T3651 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 745974 T3651 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 745977 T3651 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 745981 T3651 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 745982 T3651 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 745982 T3651 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init>

[...truncated too long message...]

it4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:414)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:383)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:370)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:273)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 	
   [junit4]   2> 900637 T3641 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/acollectionafterbaddelete/leader_elect/shard1/election/91310701157220365-core_node1-n_0000000001
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:174)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:171)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:171)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:67)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 900638 T3641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 900638 T3641 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 900638 T3641 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 900638 T3641 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 900638 T3641 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/nodes_used_collection/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Lucene45, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=ar_SA, timezone=Africa/Maseru
   [junit4]   2> NOTE: Linux 3.8.0-36-generic amd64/Oracle Corporation 1.7.0_60-ea (64-bit)/cpus=8,threads=1,free=93070648,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestClassNameShortening, OpenCloseCoreStressTest, TestIndexingPerformance, DisMaxRequestHandlerTest, MultiTermTest, CoreContainerCoreInitFailuresTest, TestRecoveryHdfs, TestSolrQueryParserDefaultOperatorResource, TestRecovery, TestFastWriter, BlockDirectoryTest, XsltUpdateRequestHandlerTest, ZkSolrClientTest, TestRandomMergePolicy, TestSolr4Spatial, BasicZkTest, TestSolrXml, FileUtilsTest, TestPartialUpdateDeduplication, DocumentAnalysisRequestHandlerTest, SimplePostToolTest, TestArbitraryIndexDir, StatsComponentTest, TestDistributedGrouping, IndexSchemaRuntimeFieldTest, TestSolrDeletionPolicy2, TestAtomicUpdateErrorCases, TestAnalyzedSuggestions, HdfsRecoveryZkTest, DirectSolrSpellCheckerTest, DirectSolrConnectionTest, TestPhraseSuggestions, TermVectorComponentDistributedTest, TestNumberUtils, DateFieldTest, TestCloudManagedSchema, TestManagedSchemaFieldResource, DirectUpdateHandlerTest, DistributedQueryComponentOptimizationTest, TestStressLucene, HighlighterTest, RAMDirectoryFactoryTest, FullSolrCloudDistribCmdsTest, TestDistribDocBasedVersion, PathHierarchyTokenizerFactoryTest, XmlUpdateRequestHandlerTest, HdfsBasicDistributedZk2Test, TestIndexSearcher, TestFieldResource, DistanceFunctionTest, TestCollationKeyRangeQueries, DeleteInactiveReplicaTest, ShardRoutingCustomTest, StandardRequestHandlerTest, AlternateDirectoryTest, TestSchemaNameResource, ChaosMonkeySafeLeaderTest, DocValuesMultiTest, TestLazyCores, HdfsUnloadDistributedZkTest, TestShardHandlerFactory, BadIndexSchemaTest, HardAutoCommitTest, HdfsSyncSliceTest, TestUpdate, FieldAnalysisRequestHandlerTest, TestSolrXmlPersistence, TestInfoStreamLogging, TestAddFieldRealTimeGet, TimeZoneUtilsTest, NoCacheHeaderTest, HighlighterConfigTest, TestNonDefinedSimilarityFactory, DistributedSuggestComponentTest, LeaderElectionIntegrationTest, SortByFunctionTest, SpellCheckComponentTest, ConnectionManagerTest, TestQuerySenderNoQuery, PreAnalyzedUpdateProcessorTest, TestDFRSimilarityFactory, UniqFieldsUpdateProcessorFactoryTest, TestCoreDiscovery, TestSolrJ, TestFoldingMultitermQuery, TestJmxIntegration, TestDistributedMissingSort, BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed on J1 in 164.03s, 1 test, 1 failure <<< FAILURES!

[...truncated 550 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:471: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:451: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:490: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1278: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:911: There were test failures: 363 suites, 1565 tests, 1 failure, 35 ignored (7 assumptions)

Total time: 46 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b04 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message