lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shalin Shekhar Mangar <shalinman...@gmail.com>
Subject Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_05) - Build # 10736 - Failure!
Date Mon, 07 Jul 2014 19:48:02 GMT
This is again due to fsync related timeouts. I am hoping that setting
forceFsync=no will fix most of these spurious problems:

[junit4]   2> 985907 T3821 oazsp.FileTxnLog.commit WARN fsync-ing the
write ahead log in SyncThread:0 took 16478ms which will adversely
effect operation latency. See the ZooKeeper troubleshooting guide



On Sun, Jul 6, 2014 at 12:00 PM, Policeman Jenkins Server <
jenkins@thetaphi.de> wrote:

> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10736/
> Java: 64bit/jdk1.8.0_05 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
>
> 2 tests failed.
> REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch
>
> Error Message:
> No live SolrServers available to handle this request
>
> Stack Trace:
> org.apache.solr.client.solrj.SolrServerException: No live SolrServers
> available to handle this request
>         at
> __randomizedtesting.SeedInfo.seed([4072EE6F5F1FAE4F:C19460772840CE73]:0)
>         at
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:317)
>         at
> org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:659)
>         at
> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
>         at
> org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
>         at
> org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1142)
>         at
> org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1111)
>         at
> org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:236)
>         at
> org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
>         at sun.reflect.GeneratedMethodAccessor52.invoke(Unknown Source)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:483)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at java.lang.Thread.run(Thread.java:745)
>
>
> REGRESSION:
>  org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch
>
> Error Message:
> Leader never changed
>
> Stack Trace:
> java.lang.AssertionError: Leader never changed
>         at
> __randomizedtesting.SeedInfo.seed([4072EE6F5F1FAE4F:C19460772840CE73]:0)
>         at org.junit.Assert.fail(Assert.java:93)
>         at
> org.apache.solr.cloud.UnloadDistributedZkTest.testCoreUnloadAndLeaders(UnloadDistributedZkTest.java:335)
>         at
> org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:75)
>         at
> org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:483)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 11487 lines...]
>    [junit4] Suite: org.apache.solr.cloud.UnloadDistributedZkTest
>    [junit4]   2> Creating dataDir:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/init-core-data-001
>    [junit4]   2> 934343 T3817
> oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext
> system property: /
>    [junit4]   2> 934346 T3817 oas.SolrTestCaseJ4.setUp ###Starting
> testDistribSearch
>    [junit4]   2> 934346 T3817 oasc.ZkTestServer.run STARTING ZK TEST SERVER
>    [junit4]   1> client port:0.0.0.0/0.0.0.0:0
>    [junit4]   2> 934347 T3818 oasc.ZkTestServer$ZKServerMain.runFromConfig
> Starting server
>    [junit4]   2> 934447 T3817 oasc.ZkTestServer.run start zk server on
> port:48973
>    [junit4]   2> 934448 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 942648 T3821 oazsp.FileTxnLog.commit WARN fsync-ing the
> write ahead log in SyncThread:0 took 8199ms which will adversely effect
> operation latency. See the ZooKeeper troubleshooting guide
>    [junit4]   2> 942650 T3824 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@27394e1
> name:ZooKeeperConnection Watcher:127.0.0.1:48973 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 942651 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 942651 T3817 oascc.SolrZkClient.makePath makePath: /solr
>    [junit4]   2> 942658 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 942660 T3826 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@7c92f475
> name:ZooKeeperConnection Watcher:127.0.0.1:48973/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 942660 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 942661 T3817 oascc.SolrZkClient.makePath makePath:
> /collections/collection1
>    [junit4]   2> 942664 T3817 oascc.SolrZkClient.makePath makePath:
> /collections/collection1/shards
>    [junit4]   2> 942668 T3817 oascc.SolrZkClient.makePath makePath:
> /collections/control_collection
>    [junit4]   2> 942669 T3817 oascc.SolrZkClient.makePath makePath:
> /collections/control_collection/shards
>    [junit4]   2> 942671 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
> to /configs/conf1/solrconfig.xml
>    [junit4]   2> 942671 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/solrconfig.xml
>    [junit4]   2> 942674 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml
> to /configs/conf1/schema.xml
>    [junit4]   2> 942674 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/schema.xml
>    [junit4]   2> 942682 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
> to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 942683 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 942685 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
> to /configs/conf1/stopwords.txt
>    [junit4]   2> 942685 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/stopwords.txt
>    [junit4]   2> 942687 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt
> to /configs/conf1/protwords.txt
>    [junit4]   2> 942687 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/protwords.txt
>    [junit4]   2> 942688 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml
> to /configs/conf1/currency.xml
>    [junit4]   2> 942689 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/currency.xml
>    [junit4]   2> 942690 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
> to /configs/conf1/enumsConfig.xml
>    [junit4]   2> 942690 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/enumsConfig.xml
>    [junit4]   2> 942692 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
> to /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 942692 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 942693 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
> to /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 942694 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 942695 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
> to /configs/conf1/old_synonyms.txt
>    [junit4]   2> 942695 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/old_synonyms.txt
>    [junit4]   2> 942696 T3817 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
> to /configs/conf1/synonyms.txt
>    [junit4]   2> 942696 T3817 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/synonyms.txt
>    [junit4]   2> 942926 T3817 oejs.Server.doStart jetty-8.1.10.v20130312
>    [junit4]   2> 942933 T3817 oejs.AbstractConnector.doStart Started
> SelectChannelConnector@127.0.0.1:54286
>    [junit4]   2> 942934 T3817 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
>    [junit4]   2> 942935 T3817 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
>    [junit4]   2> 942935 T3817 oasc.SolrResourceLoader.locateSolrHome using
> system property solr.solr.home:
> ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-002
>    [junit4]   2> 942936 T3817 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-002/'
>    [junit4]   2> 942948 T3817 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-002/solr.xml
>    [junit4]   2> 942954 T3817 oasc.CorePropertiesLocator.<init>
> Config-defined core root directory:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-002
>    [junit4]   2> 942955 T3817 oasc.CoreContainer.<init> New CoreContainer
> 1267675857
>    [junit4]   2> 942955 T3817 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-002/]
>    [junit4]   2> 942955 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 90000
>    [junit4]   2> 942956 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to:
>    [junit4]   2> 942956 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
>    [junit4]   2> 942956 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
>    [junit4]   2> 942956 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
>    [junit4]   2> 942957 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
>    [junit4]   2> 942957 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
>    [junit4]   2> 942957 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
>    [junit4]   2> 942957 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
>    [junit4]   2> 942958 T3817 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 942958 T3817 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 942958 T3817 oasc.CoreContainer.load Host Name: 127.0.0.1
>    [junit4]   2> 942958 T3817 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:48973/solr
>    [junit4]   2> 942958 T3817 oasc.ZkController.checkChrootPath zkHost
> includes chroot
>    [junit4]   2> 942959 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 942961 T3837 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@5715f840
> name:ZooKeeperConnection Watcher:127.0.0.1:48973 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 942961 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 942963 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 942964 T3839 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@786bce13
> name:ZooKeeperConnection Watcher:127.0.0.1:48973/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 942964 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 942966 T3817 oascc.SolrZkClient.makePath makePath:
> /overseer/queue
>    [junit4]   2> 942968 T3817 oascc.SolrZkClient.makePath makePath:
> /overseer/collection-queue-work
>    [junit4]   2> 942972 T3817 oascc.SolrZkClient.makePath makePath:
> /overseer/collection-map-running
>    [junit4]   2> 942973 T3817 oascc.SolrZkClient.makePath makePath:
> /overseer/collection-map-completed
>    [junit4]   2> 942975 T3817 oascc.SolrZkClient.makePath makePath:
> /overseer/collection-map-failure
>    [junit4]   2> 942978 T3817 oascc.SolrZkClient.makePath makePath:
> /live_nodes
>    [junit4]   2> 942979 T3817 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54286_
>    [junit4]   2> 942980 T3817 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:54286_
>    [junit4]   2> 942987 T3817 oascc.SolrZkClient.makePath makePath:
> /overseer_elect
>    [junit4]   2> 942990 T3817 oascc.SolrZkClient.makePath makePath:
> /overseer_elect/election
>    [junit4]   2> 942994 T3817
> oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader
> 127.0.0.1:54286_
>    [junit4]   2> 942994 T3817 oascc.SolrZkClient.makePath makePath:
> /overseer_elect/leader
>    [junit4]   2> 942997 T3817 oasc.Overseer.start Overseer
> (id=92053670881656835-127.0.0.1:54286_-n_0000000000) starting
>    [junit4]   2> 943001 T3817 oascc.SolrZkClient.makePath makePath:
> /overseer/queue-work
>    [junit4]   2> 943011 T3841 oasc.OverseerCollectionProcessor.run Process
> current queue of collection creations
>    [junit4]   2> 943011 T3817 oascc.SolrZkClient.makePath makePath:
> /clusterstate.json
>    [junit4]   2> 943014 T3817 oascc.SolrZkClient.makePath makePath:
> /aliases.json
>    [junit4]   2> 943017 T3817
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 943020 T3840 oasc.Overseer$ClusterStateUpdater.run
> Starting to work on the main queue
>    [junit4]   2> 943022 T3817 oasc.CorePropertiesLocator.discover Looking
> for core definitions underneath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-002
>    [junit4]   2> 943035 T3817 oasc.CorePropertiesLocator.discoverUnder
> Found core conf in
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-002/conf/
>    [junit4]   2> 943156 T3817 oasc.CorePropertiesLocator.discover Found 1
> core definitions
>    [junit4]   2> 943157 T3817 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
>    [junit4]   2> 943158 T3817 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
>    [junit4]   2> 943158 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 943159 T3844 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@47ebb819
> name:ZooKeeperConnection Watcher:127.0.0.1:48973/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 943159 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 943160 T3817
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 943162 T3817 oasc.ChaosMonkey.monkeyLog monkey: init -
> expire sessions:false cause connection loss:false
>    [junit4]   2> 943391 T3817
> oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
>    [junit4]   2> 943392 T3817 oejs.Server.doStart jetty-8.1.10.v20130312
>    [junit4]   2> 943394 T3817 oejs.AbstractConnector.doStart Started
> SelectChannelConnector@127.0.0.1:44978
>    [junit4]   2> 943395 T3817 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
>    [junit4]   2> 943395 T3817 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
>    [junit4]   2> 943396 T3817 oasc.SolrResourceLoader.locateSolrHome using
> system property solr.solr.home:
> ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-003
>    [junit4]   2> 943396 T3817 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-003/'
>    [junit4]   2> 943419 T3817 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-003/solr.xml
>    [junit4]   2> 943428 T3817 oasc.CorePropertiesLocator.<init>
> Config-defined core root directory:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-003
>    [junit4]   2> 943428 T3817 oasc.CoreContainer.<init> New CoreContainer
> 1791186788
>    [junit4]   2> 943428 T3817 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-003/]
>    [junit4]   2> 943429 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 90000
>    [junit4]   2> 943429 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to:
>    [junit4]   2> 943429 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
>    [junit4]   2> 943430 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
>    [junit4]   2> 943430 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
>    [junit4]   2> 943430 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
>    [junit4]   2> 943430 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
>    [junit4]   2> 943431 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
>    [junit4]   2> 943431 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
>    [junit4]   2> 943431 T3817 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 943432 T3817 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 943432 T3817 oasc.CoreContainer.load Host Name: 127.0.0.1
>    [junit4]   2> 943433 T3817 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:48973/solr
>    [junit4]   2> 943433 T3817 oasc.ZkController.checkChrootPath zkHost
> includes chroot
>    [junit4]   2> 943434 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 943435 T3855 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@24b20934
> name:ZooKeeperConnection Watcher:127.0.0.1:48973 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 943435 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 943439 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 943440 T3857 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@796659dd
> name:ZooKeeperConnection Watcher:127.0.0.1:48973/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 943440 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 943448 T3817
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 944452 T3817 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44978_
>    [junit4]   2> 944454 T3817 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:44978_
>    [junit4]   2> 944462 T3817 oasc.CorePropertiesLocator.discover Looking
> for core definitions underneath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-003
>    [junit4]   2> 944474 T3817 oasc.CorePropertiesLocator.discoverUnder
> Found core conf in
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-003/conf/
>    [junit4]   2> 944597 T3817 oasc.CorePropertiesLocator.discover Found 1
> core definitions
>    [junit4]   2> 944598 T3817 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
>    [junit4]   2> 944598 T3817 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
>    [junit4]   2> 944818 T3817
> oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
>    [junit4]   2> 944819 T3817 oejs.Server.doStart jetty-8.1.10.v20130312
>    [junit4]   2> 944821 T3817 oejs.AbstractConnector.doStart Started
> SelectChannelConnector@127.0.0.1:59618
>    [junit4]   2> 944822 T3817 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
>    [junit4]   2> 944822 T3817 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
>    [junit4]   2> 944823 T3817 oasc.SolrResourceLoader.locateSolrHome using
> system property solr.solr.home:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-004
>    [junit4]   2> 944823 T3817 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-004/'
>    [junit4]   2> 944839 T3817 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-004/solr.xml
>    [junit4]   2> 944845 T3817 oasc.CorePropertiesLocator.<init>
> Config-defined core root directory:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-004
>    [junit4]   2> 944846 T3817 oasc.CoreContainer.<init> New CoreContainer
> 1142110998
>    [junit4]   2> 944846 T3817 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-004/]
>    [junit4]   2> 944846 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 90000
>    [junit4]   2> 944847 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to:
>    [junit4]   2> 944847 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
>    [junit4]   2> 944847 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
>    [junit4]   2> 944847 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
>    [junit4]   2> 944847 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
>    [junit4]   2> 944848 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
>    [junit4]   2> 944848 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
>    [junit4]   2> 944848 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
>    [junit4]   2> 944848 T3817 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 944849 T3817 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 944849 T3817 oasc.CoreContainer.load Host Name: 127.0.0.1
>    [junit4]   2> 944849 T3817 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:48973/solr
>    [junit4]   2> 944849 T3817 oasc.ZkController.checkChrootPath zkHost
> includes chroot
>    [junit4]   2> 944850 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 944852 T3869 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@7c444ac4
> name:ZooKeeperConnection Watcher:127.0.0.1:48973 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 944852 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 944855 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 944856 T3871 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@79dcb15f
> name:ZooKeeperConnection Watcher:127.0.0.1:48973/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 944857 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 944864 T3817
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 945866 T3817 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59618_
>    [junit4]   2> 945868 T3817 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:59618_
>    [junit4]   2> 945873 T3817 oasc.CorePropertiesLocator.discover Looking
> for core definitions underneath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-004
>    [junit4]   2> 945877 T3817 oasc.CorePropertiesLocator.discoverUnder
> Found core conf in
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-004/conf/
>    [junit4]   2> 945969 T3817 oasc.CorePropertiesLocator.discover Found 1
> core definitions
>    [junit4]   2> 945970 T3817 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
>    [junit4]   2> 945970 T3817 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
>    [junit4]   2> 946194 T3817
> oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
>    [junit4]   2> 946195 T3817 oejs.Server.doStart jetty-8.1.10.v20130312
>    [junit4]   2> 946197 T3817 oejs.AbstractConnector.doStart Started
> SelectChannelConnector@127.0.0.1:34117
>    [junit4]   2> 946198 T3817 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
>    [junit4]   2> 946198 T3817 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
>    [junit4]   2> 946198 T3817 oasc.SolrResourceLoader.locateSolrHome using
> system property solr.solr.home:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-005
>    [junit4]   2> 946198 T3817 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-005/'
>    [junit4]   2> 946208 T3817 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-005/solr.xml
>    [junit4]   2> 946222 T3817 oasc.CorePropertiesLocator.<init>
> Config-defined core root directory:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-005
>    [junit4]   2> 946222 T3817 oasc.CoreContainer.<init> New CoreContainer
> 621062307
>    [junit4]   2> 946223 T3817 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-005/]
>    [junit4]   2> 946223 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 90000
>    [junit4]   2> 946223 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to:
>    [junit4]   2> 946224 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
>    [junit4]   2> 946224 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
>    [junit4]   2> 946225 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
>    [junit4]   2> 946225 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
>    [junit4]   2> 946225 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
>    [junit4]   2> 946226 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
>    [junit4]   2> 946226 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
>    [junit4]   2> 946227 T3817 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 946227 T3817 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 946227 T3817 oasc.CoreContainer.load Host Name: 127.0.0.1
>    [junit4]   2> 946228 T3817 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:48973/solr
>    [junit4]   2> 946228 T3817 oasc.ZkController.checkChrootPath zkHost
> includes chroot
>    [junit4]   2> 946229 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 946230 T3883 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@54abf20c
> name:ZooKeeperConnection Watcher:127.0.0.1:48973 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 946230 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 946235 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 946245 T3885 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@2a85dbc7
> name:ZooKeeperConnection Watcher:127.0.0.1:48973/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 946245 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 946251 T3817
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 947260 T3817 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34117_
>    [junit4]   2> 947262 T3817 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:34117_
>    [junit4]   2> 947272 T3817 oasc.CorePropertiesLocator.discover Looking
> for core definitions underneath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-005
>    [junit4]   2> 947283 T3817 oasc.CorePropertiesLocator.discoverUnder
> Found core conf in
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-005/conf/
>    [junit4]   2> 947383 T3817 oasc.CorePropertiesLocator.discover Found 1
> core definitions
>    [junit4]   2> 947384 T3817 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
>    [junit4]   2> 947385 T3817 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
>    [junit4]   2> 947614 T3817
> oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
>    [junit4]   2> 947615 T3817 oejs.Server.doStart jetty-8.1.10.v20130312
>    [junit4]   2> 947617 T3817 oejs.AbstractConnector.doStart Started
> SelectChannelConnector@127.0.0.1:39001
>    [junit4]   2> 947618 T3817 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
>    [junit4]   2> 947619 T3817 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
>    [junit4]   2> 947619 T3817 oasc.SolrResourceLoader.locateSolrHome using
> system property solr.solr.home:
> ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-006
>    [junit4]   2> 947620 T3817 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-006/'
>    [junit4]   2> 947639 T3817 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-006/solr.xml
>    [junit4]   2> 947648 T3817 oasc.CorePropertiesLocator.<init>
> Config-defined core root directory:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-006
>    [junit4]   2> 947648 T3817 oasc.CoreContainer.<init> New CoreContainer
> 518423161
>    [junit4]   2> 947648 T3817 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-006/]
>    [junit4]   2> 947649 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 90000
>    [junit4]   2> 947649 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to:
>    [junit4]   2> 947649 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
>    [junit4]   2> 947649 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
>    [junit4]   2> 947649 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
>    [junit4]   2> 947650 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
>    [junit4]   2> 947650 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
>    [junit4]   2> 947650 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
>    [junit4]   2> 947650 T3817 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
>    [junit4]   2> 947651 T3817 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 947651 T3817 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 947651 T3817 oasc.CoreContainer.load Host Name: 127.0.0.1
>    [junit4]   2> 947651 T3817 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:48973/solr
>    [junit4]   2> 947652 T3817 oasc.ZkController.checkChrootPath zkHost
> includes chroot
>    [junit4]   2> 947652 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 947654 T3897 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@ba245b4
> name:ZooKeeperConnection Watcher:127.0.0.1:48973 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 947654 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 947657 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 947658 T3899 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@72c9a51b
> name:ZooKeeperConnection Watcher:127.0.0.1:48973/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 947658 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 947662 T3817
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 948664 T3817 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39001_
>    [junit4]   2> 948666 T3817 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:39001_
>    [junit4]   2> 948676 T3817 oasc.CorePropertiesLocator.discover Looking
> for core definitions underneath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-006
>    [junit4]   2> 948690 T3817 oasc.CorePropertiesLocator.discoverUnder
> Found core conf in
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-006/conf/
>    [junit4]   2> 948832 T3817 oasc.CorePropertiesLocator.discover Found 1
> core definitions
>    [junit4]   2> 948833 T3817 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
>    [junit4]   2> 948834 T3817 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
>    [junit4]   2> 948837 T3852 oasha.CoreAdminHandler.handleCreateAction
> core create command
> dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627560899unloadcollection1_1n&name=unloadcollection1&action=CREATE&numShards=1&collection=unloadcollection&wt=javabin&version=2
>    [junit4]   2> 948840 T3852 oasc.ZkController.publish publishing
> core=unloadcollection1 state=down collection=unloadcollection
>    [junit4]   2> 948841 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 948841 T3852 oasc.ZkController.waitForCoreNodeName look
> for our core node name
>    [junit4]   2> 948843 T3840 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 948844 T3840
> oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1
> message={
>    [junit4]   2>          "operation":"state",
>    [junit4]   2>          "state":"down",
>    [junit4]   2>          "base_url":"http://127.0.0.1:44978",
>    [junit4]   2>          "core":"unloadcollection1",
>    [junit4]   2>          "roles":null,
>    [junit4]   2>          "node_name":"127.0.0.1:44978_",
>    [junit4]   2>          "shard":null,
>    [junit4]   2>          "collection":"unloadcollection",
>    [junit4]   2>          "numShards":"1",
>    [junit4]   2>          "core_node_name":null}
>    [junit4]   2> 948844 T3840
> oasc.Overseer$ClusterStateUpdater.createCollection Create collection
> unloadcollection with shards [shard1]
>    [junit4]   2> 948844 T3840
> oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard
> shard=shard1
>    [junit4]   2> 948847 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 948848 T3857 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 948848 T3899 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 948848 T3871 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 948848 T3844 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 948848 T3839 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 948848 T3885 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 949842 T3852 oasc.ZkController.waitForShardId waiting to
> find shard id in clusterstate for unloadcollection1
>    [junit4]   2> 949842 T3852 oasc.ZkController.createCollectionZkNode
> Check for collection zkNode:unloadcollection
>    [junit4]   2> 949843 T3852 oasc.ZkController.createCollectionZkNode
> Creating collection in ZooKeeper:unloadcollection
>    [junit4]   2> 949843 T3852 oasc.ZkController.getConfName Looking for
> collection configName
>    [junit4]   2> 949844 T3852 oasc.ZkController.getConfName Only one
> config set found in zk - using it:conf1
>    [junit4]   2> 949844 T3852 oascc.SolrZkClient.makePath makePath:
> /collections/unloadcollection
>    [junit4]   2> 949846 T3852 oascc.ZkStateReader.readConfigName Load
> collection config from:/collections/unloadcollection
>    [junit4]   2> 949847 T3852 oascc.ZkStateReader.readConfigName
> path=/collections/unloadcollection configName=conf1 specified config exists
> in ZooKeeper
>    [junit4]   2> 949847 T3852 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-003/unloadcollection1/'
>    [junit4]   2> 949865 T3852 oasc.SolrConfig.<init> Using Lucene
> MatchVersion: LUCENE_5_0
>    [junit4]   2> 949873 T3852 oasc.SolrConfig.<init> Loaded SolrConfig:
> solrconfig.xml
>    [junit4]   2> 949874 T3852 oass.IndexSchema.readSchema Reading Solr
> Schema from schema.xml
>    [junit4]   2> 949878 T3852 oass.IndexSchema.readSchema
> [unloadcollection1] Schema name=test
>    [junit4]   2> 949967 T3852 oass.OpenExchangeRatesOrgProvider.init
> Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 949970 T3852 oass.IndexSchema.readSchema default search
> field in schema is text
>    [junit4]   2> 949971 T3852 oass.IndexSchema.readSchema unique key
> field: id
>    [junit4]   2> 949979 T3852 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 949981 T3852 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 949982 T3852 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 949983 T3852
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
>    [junit4]   2> 949983 T3852
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
>    [junit4]   2> 949984 T3852 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 949984 T3852
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
>    [junit4]   2> 949984 T3852
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
>    [junit4]   2> 949985 T3852 oasc.CoreContainer.create Creating SolrCore
> 'unloadcollection1' using configuration from collection unloadcollection
>    [junit4]   2> 949985 T3852 oasc.SolrCore.initDirectoryFactory
> org.apache.solr.core.MockDirectoryFactory
>    [junit4]   2> 949985 T3852 oasc.SolrCore.<init> [unloadcollection1]
> Opening new SolrCore at
> ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-003/unloadcollection1/,
> dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627560899unloadcollection1_1n/
>    [junit4]   2> 949985 T3852 oasc.JmxMonitoredMap.<init> JMX monitoring
> is enabled. Adding Solr mbeans to JMX Server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@654f7f35
>    [junit4]   2> 949986 T3852 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627560899unloadcollection1_1n
>    [junit4]   2> 949986 T3852 oasc.SolrCore.getNewIndexDir New index
> directory detected: old=null
> new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627560899unloadcollection1_1n/index/
>    [junit4]   2> 949986 T3852 oasc.SolrCore.initIndex WARN
> [unloadcollection1] Solr index directory
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627560899unloadcollection1_1n/index'
> doesn't exist. Creating new index...
>    [junit4]   2> 949987 T3852 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627560899unloadcollection1_1n/index
>    [junit4]   2> 949987 T3852 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
> [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24,
> maxMergeSize=9223372036854775807,
> maxMergeSizeForForcedMerge=9223372036854775807,
> calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.20248674433591252]
>    [junit4]   2> 949988 T3852 oasc.SolrDeletionPolicy.onCommit
> SolrDeletionPolicy.onCommit: commits: num=1
>    [junit4]   2>
>  commit{dir=MockDirectoryWrapper(RAMDirectory@7048e66c
> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@30ee7058
> ),segFN=segments_1,generation=1}
>    [junit4]   2> 949988 T3852 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
>    [junit4]   2> 949990 T3852 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "nodistrib"
>    [junit4]   2> 949991 T3852 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "dedupe"
>    [junit4]   2> 949991 T3852 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "dedupe"
>    [junit4]   2> 949991 T3852 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 949991 T3852 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 949991 T3852 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>    [junit4]   2> 949992 T3852 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 949992 T3852 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 949992 T3852 oasc.SolrCore.loadUpdateProcessorChains no
> updateRequestProcessorChain defined as default, creating implicit default
>    [junit4]   2> 949993 T3852 oasc.RequestHandlers.initHandlersFromConfig
> created standard: solr.StandardRequestHandler
>    [junit4]   2> 949993 T3852 oasc.RequestHandlers.initHandlersFromConfig
> created /get: solr.RealTimeGetHandler
>    [junit4]   2> 949993 T3852 oasc.RequestHandlers.initHandlersFromConfig
> adding lazy requestHandler: solr.ReplicationHandler
>    [junit4]   2> 949993 T3852 oasc.RequestHandlers.initHandlersFromConfig
> created /replication: solr.ReplicationHandler
>    [junit4]   2> 949994 T3852 oasc.RequestHandlers.initHandlersFromConfig
> created /update: solr.UpdateRequestHandler
>    [junit4]   2> 949994 T3852 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/: org.apache.solr.handler.admin.AdminHandlers
>    [junit4]   2> 949995 T3852 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/file: solr.admin.ShowFileRequestHandler
>    [junit4]   2> 949995 T3852 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/fileedit: solr.admin.EditFileRequestHandler
>    [junit4]   2> 949997 T3852 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
>    [junit4]   2> 949998 T3852 oasu.CommitTracker.<init> Hard AutoCommit:
> disabled
>    [junit4]   2> 949998 T3852 oasu.CommitTracker.<init> Soft AutoCommit:
> disabled
>    [junit4]   2> 949999 T3852 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
> [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=25,
> maxMergedSegmentMB=43.6767578125, floorSegmentMB=0.5263671875,
> forceMergeDeletesPctAllowed=2.3967783288254263, segmentsPerTier=24.0,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
>    [junit4]   2> 949999 T3852 oasc.SolrDeletionPolicy.onInit
> SolrDeletionPolicy.onInit: commits: num=1
>    [junit4]   2>
>  commit{dir=MockDirectoryWrapper(RAMDirectory@7048e66c
> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@30ee7058
> ),segFN=segments_1,generation=1}
>    [junit4]   2> 950000 T3852 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
>    [junit4]   2> 950000 T3852 oass.SolrIndexSearcher.<init> Opening
> Searcher@2d0b2319[unloadcollection1] main
>    [junit4]   2> 950000 T3852 oascc.ZkStateReader.readConfigName Load
> collection config from:/collections/unloadcollection
>    [junit4]   2> 950001 T3852 oascc.ZkStateReader.readConfigName
> path=/collections/unloadcollection configName=conf1 specified config exists
> in ZooKeeper
>    [junit4]   2> 950001 T3852 oasr.ManagedResourceStorage.newStorageIO
> Setting up ZooKeeper-based storage for the RestManager with znodeBase:
> /configs/conf1
>    [junit4]   2> 950002 T3852
> oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
> ZooKeeperStorageIO with znodeBase: /configs/conf1
>    [junit4]   2> 950002 T3852 oasr.RestManager.init Initializing
> RestManager with initArgs: {}
>    [junit4]   2> 950002 T3852 oasr.ManagedResourceStorage.load Reading
> _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 950003 T3852
> oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data
> found for znode /configs/conf1/_rest_managed.json
>    [junit4]   2> 950003 T3852 oasr.ManagedResourceStorage.load Loaded null
> at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 950003 T3852 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/_rest_managed.json
>    [junit4]   2> 950005 T3852
> oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to
> new znode /configs/conf1/_rest_managed.json
>    [junit4]   2> 950005 T3852
> oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path
> _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 950006 T3852 oasr.RestManager.init Initializing 0
> registered ManagedResources
>    [junit4]   2> 950007 T3901 oasc.SolrCore.registerSearcher
> [unloadcollection1] Registered new searcher Searcher@2d0b2319[unloadcollection1]
> main{UninvertingDirectoryReader()}
>    [junit4]   2> 950008 T3852 oasc.CoreContainer.registerCore registering
> core: unloadcollection1
>    [junit4]   2> 950008 T3852 oasc.ZkController.register Register replica
> - core:unloadcollection1 address:http://127.0.0.1:44978
> collection:unloadcollection shard:shard1
>    [junit4]   2> 950009 T3852 oascc.SolrZkClient.makePath makePath:
> /collections/unloadcollection/leader_elect/shard1/election
>    [junit4]   2> 950013 T3852
> oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process
> for shard shard1
>    [junit4]   2> 950014 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 950014 T3852
> oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas
> found to continue.
>    [junit4]   2> 950014 T3852
> oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader -
> try and sync
>    [junit4]   2> ASYNC  NEW_CORE C3251 name=unloadcollection1
> org.apache.solr.core.SolrCore@42715fa0 url=
> http://127.0.0.1:44978/unloadcollection1 node=127.0.0.1:44978_
> C3251_STATE=coll:unloadcollection core:unloadcollection1 props:{state=down,
> base_url=http://127.0.0.1:44978, core=unloadcollection1,
> node_name=127.0.0.1:44978_}
>    [junit4]   2> 950014 T3852 C3251 P44978 oasc.SyncStrategy.sync Sync
> replicas to http://127.0.0.1:44978/unloadcollection1/
>    [junit4]   2> 950015 T3840 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 950015 T3852 C3251 P44978 oasc.SyncStrategy.syncReplicas
> Sync Success - now sync replicas to me
>    [junit4]   2> 950015 T3852 C3251 P44978 oasc.SyncStrategy.syncToMe
> http://127.0.0.1:44978/unloadcollection1/ has no replicas
>    [junit4]   2> 950016 T3852
> oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader:
> http://127.0.0.1:44978/unloadcollection1/ shard1
>    [junit4]   2> 950016 T3852 oascc.SolrZkClient.makePath makePath:
> /collections/unloadcollection/leaders/shard1
>    [junit4]   2> 950018 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 950021 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 950023 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 950125 T3844 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950125 T3871 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950125 T3839 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950125 T3857 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950125 T3885 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950125 T3899 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950171 T3852 oasc.ZkController.register We are
> http://127.0.0.1:44978/unloadcollection1/ and leader is
> http://127.0.0.1:44978/unloadcollection1/
>    [junit4]   2> 950172 T3852 oasc.ZkController.register No LogReplay
> needed for core=unloadcollection1 baseURL=http://127.0.0.1:44978
>    [junit4]   2> 950172 T3852 oasc.ZkController.checkRecovery I am the
> leader, no recovery necessary
>    [junit4]   2> 950172 T3852 oasc.ZkController.publish publishing
> core=unloadcollection1 state=active collection=unloadcollection
>    [junit4]   2> 950174 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 950174 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 950174 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 950174 T3852 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 950175 T3840 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 950175 T3852 oass.SolrDispatchFilter.handleAdminRequest
> [admin] webapp=null path=/admin/cores
> params={dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627560899unloadcollection1_1n&name=unloadcollection1&action=CREATE&numShards=1&collection=unloadcollection&wt=javabin&version=2}
> status=0 QTime=1338
>    [junit4]   2> 950175 T3840
> oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1
> message={
>    [junit4]   2>          "operation":"state",
>    [junit4]   2>          "state":"active",
>    [junit4]   2>          "base_url":"http://127.0.0.1:44978",
>    [junit4]   2>          "core":"unloadcollection1",
>    [junit4]   2>          "roles":null,
>    [junit4]   2>          "node_name":"127.0.0.1:44978_",
>    [junit4]   2>          "shard":"shard1",
>    [junit4]   2>          "collection":"unloadcollection",
>    [junit4]   2>          "numShards":"1",
>    [junit4]   2>          "core_node_name":"core_node1"}
>    [junit4]   2> 950176 T3817 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 950177 T3904 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@9e9c644
> name:ZooKeeperConnection Watcher:127.0.0.1:48973/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 950177 T3817 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 950177 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 950178 T3817
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 950179 T3817 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 950180 T3865 oasha.CoreAdminHandler.handleCreateAction
> core create command
> dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627562243unloadcollection1_2n&name=unloadcollection2&action=CREATE&collection=unloadcollection&wt=javabin&version=2
>    [junit4]   2> 950182 T3865 oasc.ZkController.publish publishing
> core=unloadcollection2 state=down collection=unloadcollection
>    [junit4]   2> 950182 T3865 oasc.ZkController.publish numShards not
> found on descriptor - reading it from system property
>    [junit4]   2> 950183 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 950183 T3865 oasc.ZkController.waitForCoreNodeName look
> for our core node name
>    [junit4]   2> 950183 T3840
> oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2
> message={
>    [junit4]   2>          "operation":"state",
>    [junit4]   2>          "state":"down",
>    [junit4]   2>          "base_url":"http://127.0.0.1:59618",
>    [junit4]   2>          "core":"unloadcollection2",
>    [junit4]   2>          "roles":null,
>    [junit4]   2>          "node_name":"127.0.0.1:59618_",
>    [junit4]   2>          "shard":null,
>    [junit4]   2>          "collection":"unloadcollection",
>    [junit4]   2>          "numShards":"2",
>    [junit4]   2>          "core_node_name":null}
>    [junit4]   2> 950184 T3840
> oasc.Overseer$ClusterStateUpdater.updateState Collection already exists
> with numShards=1
>    [junit4]   2> 950184 T3840
> oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard
> shard=shard1
>    [junit4]   2> 950185 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 950287 T3839 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950287 T3904 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950287 T3885 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950287 T3844 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950287 T3857 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950287 T3899 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 950288 T3871 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 951183 T3865 oasc.ZkController.waitForShardId waiting to
> find shard id in clusterstate for unloadcollection2
>    [junit4]   2> 951184 T3865 oasc.ZkController.createCollectionZkNode
> Check for collection zkNode:unloadcollection
>    [junit4]   2> 951185 T3865 oasc.ZkController.createCollectionZkNode
> Collection zkNode exists
>    [junit4]   2> 951186 T3865 oascc.ZkStateReader.readConfigName Load
> collection config from:/collections/unloadcollection
>    [junit4]   2> 951187 T3865 oascc.ZkStateReader.readConfigName
> path=/collections/unloadcollection configName=conf1 specified config exists
> in ZooKeeper
>    [junit4]   2> 951188 T3865 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-004/unloadcollection2/'
>    [junit4]   2> 951217 T3865 oasc.SolrConfig.<init> Using Lucene
> MatchVersion: LUCENE_5_0
>    [junit4]   2> 951232 T3865 oasc.SolrConfig.<init> Loaded SolrConfig:
> solrconfig.xml
>    [junit4]   2> 951234 T3865 oass.IndexSchema.readSchema Reading Solr
> Schema from schema.xml
>    [junit4]   2> 951242 T3865 oass.IndexSchema.readSchema
> [unloadcollection2] Schema name=test
>    [junit4]   2> 951329 T3865 oass.OpenExchangeRatesOrgProvider.init
> Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 951332 T3865 oass.IndexSchema.readSchema default search
> field in schema is text
>    [junit4]   2> 951333 T3865 oass.IndexSchema.readSchema unique key
> field: id
>    [junit4]   2> 951342 T3865 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 951344 T3865 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 951345 T3865 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 951346 T3865
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
>    [junit4]   2> 951348 T3865
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
>    [junit4]   2> 951348 T3865 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 951348 T3865
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
>    [junit4]   2> 951349 T3865
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
>    [junit4]   2> 951349 T3865 oasc.CoreContainer.create Creating SolrCore
> 'unloadcollection2' using configuration from collection unloadcollection
>    [junit4]   2> 951349 T3865 oasc.SolrCore.initDirectoryFactory
> org.apache.solr.core.MockDirectoryFactory
>    [junit4]   2> 951349 T3865 oasc.SolrCore.<init> [unloadcollection2]
> Opening new SolrCore at
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-004/unloadcollection2/,
> dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627562243unloadcollection1_2n/
>    [junit4]   2> 951350 T3865 oasc.JmxMonitoredMap.<init> JMX monitoring
> is enabled. Adding Solr mbeans to JMX Server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@654f7f35
>    [junit4]   2> 951350 T3865 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627562243unloadcollection1_2n
>    [junit4]   2> 951350 T3865 oasc.SolrCore.getNewIndexDir New index
> directory detected: old=null
> new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627562243unloadcollection1_2n/index/
>    [junit4]   2> 951351 T3865 oasc.SolrCore.initIndex WARN
> [unloadcollection2] Solr index directory
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627562243unloadcollection1_2n/index'
> doesn't exist. Creating new index...
>    [junit4]   2> 951351 T3865 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627562243unloadcollection1_2n/index
>    [junit4]   2> 951351 T3865 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy:
> [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24,
> maxMergeSize=9223372036854775807,
> maxMergeSizeForForcedMerge=9223372036854775807,
> calibrateSizeByDeletes=true, maxMergeDocs=2147483647,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.20248674433591252]
>    [junit4]   2> 951352 T3865 oasc.SolrDeletionPolicy.onCommit
> SolrDeletionPolicy.onCommit: commits: num=1
>    [junit4]   2>
>  commit{dir=MockDirectoryWrapper(RAMDirectory@1ab0159
> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4a774e26
> ),segFN=segments_1,generation=1}
>    [junit4]   2> 951352 T3865 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
>    [junit4]   2> 951354 T3865 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "nodistrib"
>    [junit4]   2> 951355 T3865 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "dedupe"
>    [junit4]   2> 951355 T3865 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "dedupe"
>    [junit4]   2> 951355 T3865 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 951355 T3865 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 951356 T3865 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>    [junit4]   2> 951356 T3865 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 951356 T3865 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 951356 T3865 oasc.SolrCore.loadUpdateProcessorChains no
> updateRequestProcessorChain defined as default, creating implicit default
>    [junit4]   2> 951357 T3865 oasc.RequestHandlers.initHandlersFromConfig
> created standard: solr.StandardRequestHandler
>    [junit4]   2> 951357 T3865 oasc.RequestHandlers.initHandlersFromConfig
> created /get: solr.RealTimeGetHandler
>    [junit4]   2> 951357 T3865 oasc.RequestHandlers.initHandlersFromConfig
> adding lazy requestHandler: solr.ReplicationHandler
>    [junit4]   2> 951357 T3865 oasc.RequestHandlers.initHandlersFromConfig
> created /replication: solr.ReplicationHandler
>    [junit4]   2> 951358 T3865 oasc.RequestHandlers.initHandlersFromConfig
> created /update: solr.UpdateRequestHandler
>    [junit4]   2> 951358 T3865 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/: org.apache.solr.handler.admin.AdminHandlers
>    [junit4]   2> 951358 T3865 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/file: solr.admin.ShowFileRequestHandler
>    [junit4]   2> 951359 T3865 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/fileedit: solr.admin.EditFileRequestHandler
>    [junit4]   2> 951360 T3865 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
>    [junit4]   2> 951361 T3865 oasu.CommitTracker.<init> Hard AutoCommit:
> disabled
>    [junit4]   2> 951361 T3865 oasu.CommitTracker.<init> Soft AutoCommit:
> disabled
>    [junit4]   2> 951361 T3865 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
> [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=25,
> maxMergedSegmentMB=43.6767578125, floorSegmentMB=0.5263671875,
> forceMergeDeletesPctAllowed=2.3967783288254263, segmentsPerTier=24.0,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
>    [junit4]   2> 951362 T3865 oasc.SolrDeletionPolicy.onInit
> SolrDeletionPolicy.onInit: commits: num=1
>    [junit4]   2>
>  commit{dir=MockDirectoryWrapper(RAMDirectory@1ab0159
> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4a774e26
> ),segFN=segments_1,generation=1}
>    [junit4]   2> 951362 T3865 oasc.SolrDeletionPolicy.updateCommits newest
> commit generation = 1
>    [junit4]   2> 951362 T3865 oass.SolrIndexSearcher.<init> Opening
> Searcher@12ac6621[unloadcollection2] main
>    [junit4]   2> 951363 T3865 oascc.ZkStateReader.readConfigName Load
> collection config from:/collections/unloadcollection
>    [junit4]   2> 951363 T3865 oascc.ZkStateReader.readConfigName
> path=/collections/unloadcollection configName=conf1 specified config exists
> in ZooKeeper
>    [junit4]   2> 951364 T3865 oasr.ManagedResourceStorage.newStorageIO
> Setting up ZooKeeper-based storage for the RestManager with znodeBase:
> /configs/conf1
>    [junit4]   2> 951364 T3865
> oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured
> ZooKeeperStorageIO with znodeBase: /configs/conf1
>    [junit4]   2> 951364 T3865 oasr.RestManager.init Initializing
> RestManager with initArgs: {}
>    [junit4]   2> 951365 T3865 oasr.ManagedResourceStorage.load Reading
> _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 951365 T3865
> oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38
> bytes from znode /configs/conf1/_rest_managed.json
>    [junit4]   2> 951366 T3865 oasr.ManagedResourceStorage.load Loaded
> LinkedHashMap at path _rest_managed.json using
> ZooKeeperStorageIO:path=/configs/conf1
>    [junit4]   2> 951366 T3865 oasr.ManagedResource.reloadFromStorage
> Loaded initArgs {} for /rest/managed
>    [junit4]   2> 951366 T3865 oasr.RestManager.init Initializing 0
> registered ManagedResources
>    [junit4]   2> 951367 T3905 oasc.SolrCore.registerSearcher
> [unloadcollection2] Registered new searcher Searcher@12ac6621[unloadcollection2]
> main{UninvertingDirectoryReader()}
>    [junit4]   2> 951368 T3865 oasc.CoreContainer.registerCore registering
> core: unloadcollection2
>    [junit4]   2> 951368 T3865 oasc.ZkController.register Register replica
> - core:unloadcollection2 address:http://127.0.0.1:59618
> collection:unloadcollection shard:shard1
>    [junit4]   2> 951370 T3865 oasc.ZkController.register We are
> http://127.0.0.1:59618/unloadcollection2/ and leader is
> http://127.0.0.1:44978/unloadcollection1/
>    [junit4]   2> 951370 T3865 oasc.ZkController.register No LogReplay
> needed for core=unloadcollection2 baseURL=http://127.0.0.1:59618
>    [junit4]   2> 951370 T3865 oasc.ZkController.checkRecovery Core needs
> to recover:unloadcollection2
>    [junit4]   2> 951371 T3865 oasu.DefaultSolrCoreState.doRecovery Running
> recovery - first canceling any ongoing recovery
>    [junit4]   2> ASYNC  NEW_CORE C3252 name=unloadcollection2
> org.apache.solr.core.SolrCore@17e98ef url=
> http://127.0.0.1:59618/unloadcollection2 node=127.0.0.1:59618_
> C3252_STATE=coll:unloadcollection core:unloadcollection2 props:{state=down,
> base_url=http://127.0.0.1:59618, core=unloadcollection2,
> node_name=127.0.0.1:59618_}
>    [junit4]   2> 951371 T3907 C3252 P59618 oasc.RecoveryStrategy.run
> Starting recovery process.  core=unloadcollection2
> recoveringAfterStartup=true
>    [junit4]   2> 951371 T3865 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 951372 T3907 C3252 P59618
> oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
>    [junit4]   2> 951372 T3907 C3252 P59618
> oasc.RecoveryStrategy.doRecovery Publishing state of core unloadcollection2
> as recovering, leader is http://127.0.0.1:44978/unloadcollection1/ and I
> am http://127.0.0.1:59618/unloadcollection2/
>    [junit4]   2> 951372 T3907 C3252 P59618 oasc.ZkController.publish
> publishing core=unloadcollection2 state=recovering
> collection=unloadcollection
>    [junit4]   2> 951372 T3865 oass.SolrDispatchFilter.handleAdminRequest
> [admin] webapp=null path=/admin/cores
> params={dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.UnloadDistributedZkTest-4072EE6F5F1FAE4F-001/tempDir-007/1404627562243unloadcollection1_2n&name=unloadcollection2&action=CREATE&collection=unloadcollection&wt=javabin&version=2}
> status=0 QTime=1192
>    [junit4]   2> 951372 T3907 C3252 P59618 oasc.ZkController.publish
> numShards not found on descriptor - reading it from system property
>    [junit4]   2> 951373 T3817 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 951374 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 951374 T3817
> oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for
> recoveries to finish - collection: unloadcollection failOnTimeout:true
> timeout (sec):330
>    [junit4]   2> 951374 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 951376 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 951374 T3907 C3252 P59618
> oasc.RecoveryStrategy.sendPrepRecoveryCmd Sending prep recovery command to
> http://127.0.0.1:44978; WaitForState:
> action=PREPRECOVERY&core=unloadcollection1&nodeName=127.0.0.1%3A59618_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
>    [junit4]   2> 951376 T3817 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 951377 T3851
> oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for
> coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader:
> true, onlyIfLeaderActive: true
>    [junit4]   2> 951377 T3840 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 951377 T3851 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 951378 T3840
> oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2
> message={
>    [junit4]   2>          "operation":"state",
>    [junit4]   2>          "state":"recovering",
>    [junit4]   2>          "base_url":"http://127.0.0.1:59618",
>    [junit4]   2>          "core":"unloadcollection2",
>    [junit4]   2>          "roles":null,
>    [junit4]   2>          "node_name":"127.0.0.1:59618_",
>    [junit4]   2>          "shard":"shard1",
>    [junit4]   2>          "collection":"unloadcollection",
>    [junit4]   2>          "numShards":"2",
>    [junit4]   2>          "core_node_name":"core_node2"}
>    [junit4]   2> 951378 T3851
> oasha.CoreAdminHandler.handleWaitForStateAction Will wait a max of 183
> seconds to see unloadcollection1 (shard1 of unloadcollection) have state:
> recovering
>    [junit4]   2> 951378 T3851
> oasha.CoreAdminHandler.handleWaitForStateAction In
> WaitForState(recovering): collection=unloadcollection, shard=shard1,
> thisCore=unloadcollection1, leaderDoesNotNeedRecovery=false, isLeader?
> true, live=true, checkLive=true, currentState=down, localState=active,
> nodeName=127.0.0.1:59618_, coreNodeName=core_node2,
> onlyIfActiveCheckResult=false, nodeProps:
> core_node2:{"state":"down","base_url":"http://127.0.0.1:59618
> ","core":"unloadcollection2","node_name":"127.0.0.1:59618_"}
>    [junit4]   2> 951379 T3839
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 951481 T3899 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 951482 T3871 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 951481 T3904 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 951481 T3885 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 951481 T3857 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 951481 T3844 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 951481 T3839 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 952378 T3817 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2> 952379 T3851
> oasha.CoreAdminHandler.handleWaitForStateAction In
> WaitForState(recovering): collection=unloadcollection, shard=shard1,
> thisCore=unloadcollection1, leaderDoesNotNeedRecovery=false, isLeader?
> true, live=true, checkLive=true, currentState=recovering,
> localState=active, nodeName=127.0.0.1:59618_, coreNodeName=core_node2,
> onlyIfActiveCheckResult=false, nodeProps:
> core_node2:{"state":"recovering","base_url":"http://127.0.0.1:59618
> ","core":"unloadcollection2","node_name":"127.0.0.1:59618_"}
>    [junit4]   2> 952379 T3851
> oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName:
> core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1
> seconds.
>    [junit4]   2> 952379 T3851 oass.SolrDispatchFilter.handleAdminRequest
> [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:59618_&onlyIfLeaderActive=true&core=unloadcollection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2}
> status=0 QTime=1002
>    [junit4]   2> 953379 T3817 oascc.ZkStateReader.updateClusterState
> Updating cloud state from ZooKeeper...
>    [junit4]   2>  C3252_STATE=coll:unloadcollection core:unloadcollection2
> props:{state=recovering, base_url=http://127.0.0.1:59618,
> core=unloadcollection2, node_name=127.0.0.1:59618_}
>    [junit4]   2> 954380 T3907 C3252 P59618
> oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from
> http://127.0.0.1:44978/unloadcollection1/ core=unloadcollection2 -
> recoveringAfterStartup=true
>    [junit4]   2> 954380 T3907 C3252 P59618 oasu.PeerSync.sync PeerSync:
> core=unloadcollection2 u
>
> [...truncated too long message...]
>
> ion.java:127)
>    [junit4]   2>        at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>    [junit4]   2>        at
> org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
>    [junit4]   2>        at
> org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:243)
>    [junit4]   2>        at
> org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:240)
>    [junit4]   2>        at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
>    [junit4]   2>        at
> org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:240)
>    [junit4]   2>        at
> org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:346)
>    [junit4]   2>        at
> org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:330)
>    [junit4]   2>        at
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:198)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
>    [junit4]   2>        at
> org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:452)
>    [junit4]   2>        at
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:216)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
>    [junit4]   2>        at
> org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:452)
>    [junit4]   2>        at
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:216)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
>    [junit4]   2>        at
> org.apache.solr.cloud.ShardLeaderElectionContext.rejoinLeaderElection(ElectionContext.java:452)
>    [junit4]   2>        at
> org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:216)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
>    [junit4]   2>        at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
>    [junit4]   2>        at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
>    [junit4]   2>
>    [junit4]   2> 1042980 T4036 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
>    [junit4]   2> 1042980 T4036 oascc.ZkStateReader$2.process WARN
> ZooKeeper watch triggered, but Solr cannot talk to ZK
>    [junit4]   2> 1042980 T4036 oascc.ZkStateReader$3.process WARN
> ZooKeeper watch triggered, but Solr cannot talk to ZK
>    [junit4]   2> 1042981 T4036 oasc.LeaderElector$ElectionWatcher.process
> WARN  org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/election
>    [junit4]   2>        at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
>    [junit4]   2>        at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>    [junit4]   2>        at
> org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
>    [junit4]   2>        at
> org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
>    [junit4]   2>        at
> org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
>    [junit4]   2>        at
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
>    [junit4]   2>        at
> org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
>    [junit4]   2>        at
> org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
>    [junit4]   2>        at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
>    [junit4]   2>        at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
>    [junit4]   2>
>    [junit4]   2> NOTE: leaving temporary files on disk at:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.SyncSliceTest-4072EE6F5F1FAE4F-001
>    [junit4]   2> NOTE: test params are: codec=Lucene49: {rnd_b=FSTOrd41,
> other_tl1=PostingsFormat(name=Direct),
> _version_=PostingsFormat(name=Direct),
> multiDefault=PostingsFormat(name=Lucene41WithOrds),
> a_t=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Direct),
> a_i1=PostingsFormat(name=Direct), id=FSTOrd41,
> text=PostingsFormat(name=Memory doPackFST= true),
> range_facet_l=PostingsFormat(name=Direct),
> timestamp=PostingsFormat(name=Direct)},
> docValues:{timestamp=DocValuesFormat(name=Memory)},
> sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=in_ID,
> timezone=Asia/Bahrain
>    [junit4]   2> NOTE: Linux 3.8.0-42-generic amd64/Oracle Corporation
> 1.8.0_05 (64-bit)/cpus=8,threads=1,free=118877464,total=258912256
>    [junit4]   2> NOTE: All tests run in this JVM: [SoftAutoCommitTest,
> SolrTestCaseJ4Test, TestFiltering, TestSimpleQParserPlugin,
> CollectionsAPIAsyncDistributedZkTest, CacheHeaderTest, ResponseHeaderTest,
> CoreMergeIndexesAdminHandlerTest, SynonymTokenizerTest,
> DistributedQueryComponentOptimizationTest, CopyFieldTest,
> HdfsChaosMonkeySafeLeaderTest, TestJoin, SortByFunctionTest,
> HdfsBasicDistributedZkTest, OutputWriterTest, TestDocumentBuilder,
> TestIndexSearcher, TestSystemIdResolver, DirectSolrConnectionTest,
> SolrCoreTest, BJQParserTest, TestZkChroot, DeleteReplicaTest,
> SpatialFilterTest, TermVectorComponentTest, TestCoreContainer,
> DistributedSuggestComponentTest, DocValuesTest,
> TestOverriddenPrefixQueryForCustomFieldType, SampleTest, NumericFieldsTest,
> TestSuggestSpellingConverter, TestRequestStatusCollectionAPI,
> AliasIntegrationTest, TestDynamicFieldResource, ResourceLoaderTest,
> TestLeaderElectionZkExpiry, MultiThreadedOCPTest, LoggingHandlerTest,
> URLClassifyProcessorTest, TestPHPSerializedResponseWriter,
> DebugComponentTest, AddBlockUpdateTest, DirectUpdateHandlerOptimizeTest,
> SOLR749Test, HdfsLockFactoryTest, TestBadConfig, TestSolrXmlPersistence,
> TestOmitPositions, DeleteShardTest, TestSerializedLuceneMatchVersion,
> FieldAnalysisRequestHandlerTest, TestReversedWildcardFilterFactory,
> DOMUtilTest, TestComplexPhraseQParserPlugin,
> CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest,
> TestCollectionAPI, TestRealTimeGet, TestManagedResourceStorage,
> TestSolrXml, SolrIndexConfigTest, LegacyHTMLStripCharFilterTest,
> MultiTermTest, AddSchemaFieldsUpdateProcessorFactoryTest, EnumFieldTest,
> TestUniqueKeyFieldResource, TestPerFieldSimilarity, TestValueSourceCache,
> TestSolrQueryParserResource, ReplicationFactorTest, TestLRUCache,
> SuggesterFSTTest, TestLFUCache, RangeFacetTest, SearchHandlerTest,
> OverseerStatusTest, TestMultiCoreConfBootstrap, AutoCommitTest,
> TestQuerySenderNoQuery, ModifyConfFileTest, TestNonNRTOpen,
> DefaultValueUpdateProcessorTest, TestChildDocTransformer,
> TestSolrXMLSerializer, TestStandardQParsers, SliceStateTest,
> DistributedDebugComponentTest, AbstractAnalyticsFacetTest,
> SolrRequestParserTest, ShardRoutingCustomTest, TestIBSimilarityFactory,
> TestRecovery, ExpressionTest, TestRandomFaceting,
> DistributedExpandComponentTest, TestRandomMergePolicy,
> UniqFieldsUpdateProcessorFactoryTest, ChaosMonkeySafeLeaderTest,
> BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest]
>    [junit4] Completed on J1 in 54.05s, 1 test, 1 error <<< FAILURES!
>
> [...truncated 678 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1298:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:922:
> There were test failures: 416 suites, 1706 tests, 1 error, 1 failure, 38
> ignored (17 assumptions)
>
> Total time: 73 minutes 6 seconds
> Build step 'Invoke Ant' marked build as failure
> [description-setter] Description set: Java: 64bit/jdk1.8.0_05
> -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>



-- 
Regards,
Shalin Shekhar Mangar.

Mime
View raw message