lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 1087 - Failure
Date Thu, 21 Mar 2013 11:39:44 GMT
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/1087/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch

Error Message:
shard3 is not consistent.  Got 13 from http://127.0.0.1:18886/pc_kzu/vk/collection1lastClient and got 0 from http://127.0.0.1:18917/pc_kzu/vk/collection1

Stack Trace:
java.lang.AssertionError: shard3 is not consistent.  Got 13 from http://127.0.0.1:18886/pc_kzu/vk/collection1lastClient and got 0 from http://127.0.0.1:18917/pc_kzu/vk/collection1
	at __randomizedtesting.SeedInfo.seed([8ABE485330C4961E:B58C64B479BF622]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:963)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:137)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:44916/b/rt/collection1 Dead Guy:http://127.0.0.1:44912/b/rt/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:44927/b/rt/collection1], CloudJettyRunner [url=http://127.0.0.1:44927/b/rt/collection1]]

Stack Trace:
java.lang.AssertionError: Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:44916/b/rt/collection1 Dead Guy:http://127.0.0.1:44912/b/rt/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:44927/b/rt/collection1], CloudJettyRunner [url=http://127.0.0.1:44927/b/rt/collection1]]
	at __randomizedtesting.SeedInfo.seed([8ABE485330C4961E:B58C64B479BF622]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:212)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 8955 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4:junit4]   2> 0 T23 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /pc_kzu/vk
[junit4:junit4]   2> 18 T23 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeySafeLeaderTest-1363864687969
[junit4:junit4]   2> 25 T23 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 31 T24 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 94 T24 oaz.Environment.logEnv Server environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
[junit4:junit4]   2> 94 T24 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 94 T24 oaz.Environment.logEnv Server environment:java.version=1.7.0_17
[junit4:junit4]   2> 95 T24 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 95 T24 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 95 T24 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/lucene-codecs-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-13.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/objenesis-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/org.restlet-2.1.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/org.restlet.ext.servlet-2.1.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.2.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.2.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/noggit-0.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
[junit4:junit4]   2> 97 T24 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 97 T24 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 97 T24 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 97 T24 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 98 T24 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 98 T24 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 98 T24 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 98 T24 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 99 T24 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 136 T24 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 137 T24 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 137 T24 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 144 T24 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 232 T23 oasc.ZkTestServer.run start zk server on port:58321
[junit4:junit4]   2> 239 T24 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 329 T23 oaz.Environment.logEnv Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
[junit4:junit4]   2> 330 T23 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 331 T23 oaz.Environment.logEnv Client environment:java.version=1.7.0_17
[junit4:junit4]   2> 331 T23 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 332 T23 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 333 T23 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/lucene-codecs-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.3-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-13.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/objenesis-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/org.restlet-2.1.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/org.restlet.ext.servlet-2.1.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.2.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.2.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/noggit-0.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
[junit4:junit4]   2> 335 T23 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 335 T23 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 336 T23 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 337 T23 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 337 T23 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 338 T23 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 338 T23 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 339 T23 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 339 T23 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 345 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@2c72e086
[junit4:junit4]   2> 430 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 438 T29 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 441 T29 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 441 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:50236
[junit4:junit4]   2> 473 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:50236
[junit4:junit4]   2> 488 T27 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 529 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab96670000 with negotiated timeout 10000 for client /140.211.11.196:50236
[junit4:junit4]   2> 529 T29 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab96670000, negotiated timeout = 10000
[junit4:junit4]   2> 536 T30 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c72e086 name:ZooKeeperConnection Watcher:127.0.0.1:58321 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 536 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 541 T23 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 623 T28 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8cab96670000
[junit4:junit4]   2> 634 T30 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 634 T23 oaz.ZooKeeper.close Session: 0x13d8cab96670000 closed
[junit4:junit4]   2> 635 T25 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:50236 which had sessionid 0x13d8cab96670000
[junit4:junit4]   2> 635 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@3d5045ad
[junit4:junit4]   2> 636 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 637 T31 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 638 T31 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 638 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31207
[junit4:junit4]   2> 639 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31207
[junit4:junit4]   2> 640 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab96670001 with negotiated timeout 10000 for client /140.211.11.196:31207
[junit4:junit4]   2> 641 T31 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab96670001, negotiated timeout = 10000
[junit4:junit4]   2> 641 T32 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d5045ad name:ZooKeeperConnection Watcher:127.0.0.1:58321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 642 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 656 T23 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 686 T23 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 726 T23 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 731 T23 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 789 T23 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 791 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 830 T23 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 832 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 942 T23 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 943 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 948 T23 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 949 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 954 T23 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 955 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 960 T23 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 961 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 966 T23 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 967 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 972 T23 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 973 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 978 T23 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 979 T23 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 984 T28 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8cab96670001
[junit4:junit4]   2> 985 T32 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 985 T23 oaz.ZooKeeper.close Session: 0x13d8cab96670001 closed
[junit4:junit4]   2> 985 T25 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:31207 which had sessionid 0x13d8cab96670001
[junit4:junit4]   2> 1766 T23 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 2011 T23 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:32209
[junit4:junit4]   2> 2049 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2050 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2050 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1363864688936
[junit4:junit4]   2> 2051 T23 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1363864688936/solr.xml
[junit4:junit4]   2> 2051 T23 oasc.CoreContainer.<init> New CoreContainer 2026758914
[junit4:junit4]   2> 2052 T23 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1363864688936/'
[junit4:junit4]   2> 2052 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1363864688936/'
[junit4:junit4]   2> 2164 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 2165 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2165 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 2165 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2166 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2166 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2167 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2167 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2167 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2215 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 2688 T23 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2708 T23 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58321/solr
[junit4:junit4]   2> 2731 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 2731 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1642d13e
[junit4:junit4]   2> 2732 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2734 T43 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 2734 T43 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 2735 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63866
[junit4:junit4]   2> 2735 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63866
[junit4:junit4]   2> 2738 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab96670002 with negotiated timeout 20000 for client /140.211.11.196:63866
[junit4:junit4]   2> 2738 T43 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab96670002, negotiated timeout = 20000
[junit4:junit4]   2> 2739 T44 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1642d13e name:ZooKeeperConnection Watcher:127.0.0.1:58321 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2739 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2741 T28 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8cab96670002
[junit4:junit4]   2> 2743 T25 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63866 which had sessionid 0x13d8cab96670002
[junit4:junit4]   2> 2743 T44 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2743 T23 oaz.ZooKeeper.close Session: 0x13d8cab96670002 closed
[junit4:junit4]   2> 2747 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 2761 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@60a7d346
[junit4:junit4]   2> 2762 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2763 T45 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 2763 T45 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 2764 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12108
[junit4:junit4]   2> 2764 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12108
[junit4:junit4]   2> 2766 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab96670003 with negotiated timeout 20000 for client /140.211.11.196:12108
[junit4:junit4]   2> 2766 T45 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab96670003, negotiated timeout = 20000
[junit4:junit4]   2> 2766 T46 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60a7d346 name:ZooKeeperConnection Watcher:127.0.0.1:58321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2767 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2776 T23 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 2787 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670003 type:create cxid:0x7 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2789 T23 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 2804 T23 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 2808 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32209_pc_kzu%2Fvk
[junit4:junit4]   2> 2812 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670003 type:delete cxid:0x12 zxid:0x1e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:32209_pc_kzu%2Fvk Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:32209_pc_kzu%2Fvk
[junit4:junit4]   2> 2814 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32209_pc_kzu%2Fvk
[junit4:junit4]   2> 2819 T23 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 2839 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670003 type:delete cxid:0x20 zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 2841 T23 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 2845 T23 oasc.Overseer.start Overseer (id=89382236207906819-127.0.0.1:32209_pc_kzu%2Fvk-n_0000000000) starting
[junit4:junit4]   2> 2846 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670003 type:create cxid:0x25 zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2858 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670003 type:create cxid:0x26 zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2860 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670003 type:create cxid:0x28 zxid:0x27 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2862 T23 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 2880 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670003 type:create cxid:0x2e zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2883 T48 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 2883 T23 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 2886 T23 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 2889 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2908 T47 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 2941 T49 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1363864688936/collection1
[junit4:junit4]   2> 2941 T49 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 2942 T49 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 2943 T49 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 2945 T49 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1363864688936/collection1/'
[junit4:junit4]   2> 2946 T49 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1363864688936/collection1/lib/README' to classloader
[junit4:junit4]   2> 2947 T49 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1363864688936/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3027 T49 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 3128 T49 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 3229 T49 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3247 T49 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 3818 T49 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 3831 T49 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 3834 T49 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3869 T49 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3875 T49 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3880 T49 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3883 T49 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 3883 T49 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 3883 T49 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3885 T49 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 3886 T49 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 3886 T49 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 3914 T49 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1363864688936/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/control/data/
[junit4:junit4]   2> 3918 T49 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@636323cc
[junit4:junit4]   2> 3919 T49 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 3920 T49 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/control/data forceNew: false
[junit4:junit4]   2> 3920 T49 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/control/data/index/
[junit4:junit4]   2> 3921 T49 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3922 T49 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/control/data/index forceNew: false
[junit4:junit4]   2> 3927 T49 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/control/data/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3927 T49 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 3973 T49 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 3974 T49 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 3974 T49 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 3975 T49 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3976 T49 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3976 T49 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 3977 T49 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 3978 T49 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3978 T49 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3988 T49 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3995 T49 oass.SolrIndexSearcher.<init> Opening Searcher@6b7d709f main
[junit4:junit4]   2> 4015 T49 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4015 T49 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4019 T50 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b7d709f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4021 T49 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 4021 T49 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4417 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4418 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32209_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32209/pc_kzu/vk"}
[junit4:junit4]   2> 4418 T47 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 4431 T47 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 4452 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 5025 T49 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 5025 T49 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:32209/pc_kzu/vk collection:control_collection shard:shard1
[junit4:junit4]   2> 5043 T49 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 5066 T49 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 5095 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670003 type:delete cxid:0x82 zxid:0x35 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 5097 T49 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 5098 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670003 type:create cxid:0x83 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5109 T49 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 5109 T49 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 5115 T49 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:32209/pc_kzu/vk/collection1/
[junit4:junit4]   2> 5116 T49 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 5116 T49 oasc.SyncStrategy.syncToMe http://127.0.0.1:32209/pc_kzu/vk/collection1/ has no replicas
[junit4:junit4]   2> 5117 T49 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:32209/pc_kzu/vk/collection1/
[junit4:junit4]   2> 5117 T49 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 5125 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670003 type:create cxid:0x8e zxid:0x3a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5959 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5983 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 6004 T49 oasc.ZkController.register We are http://127.0.0.1:32209/pc_kzu/vk/collection1/ and leader is http://127.0.0.1:32209/pc_kzu/vk/collection1/
[junit4:junit4]   2> 6007 T49 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32209/pc_kzu/vk
[junit4:junit4]   2> 6007 T49 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6008 T49 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6008 T49 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6011 T49 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6016 T23 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 6016 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6073 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 6092 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 6096 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4099dde9
[junit4:junit4]   2> 6097 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6098 T52 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 6098 T52 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 6098 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:29805
[junit4:junit4]   2> 6099 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:29805
[junit4:junit4]   2> 6101 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab96670004 with negotiated timeout 10000 for client /140.211.11.196:29805
[junit4:junit4]   2> 6101 T52 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab96670004, negotiated timeout = 10000
[junit4:junit4]   2> 6102 T53 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4099dde9 name:ZooKeeperConnection Watcher:127.0.0.1:58321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6103 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6105 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 6110 T23 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 6317 T23 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 6320 T23 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:32591
[junit4:junit4]   2> 6321 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 6322 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6323 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1363864694060
[junit4:junit4]   2> 6323 T23 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1363864694060/solr.xml
[junit4:junit4]   2> 6324 T23 oasc.CoreContainer.<init> New CoreContainer 1145788263
[junit4:junit4]   2> 6325 T23 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1363864694060/'
[junit4:junit4]   2> 6325 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1363864694060/'
[junit4:junit4]   2> 6400 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 6401 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 6401 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 6402 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 6403 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 6403 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 6404 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 6404 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 6405 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 6406 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 6433 T23 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 6457 T23 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58321/solr
[junit4:junit4]   2> 6458 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 6459 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1020c592
[junit4:junit4]   2> 6460 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6461 T63 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 6462 T63 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 6463 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:46382
[junit4:junit4]   2> 6463 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:46382
[junit4:junit4]   2> 6466 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab96670005 with negotiated timeout 20000 for client /140.211.11.196:46382
[junit4:junit4]   2> 6466 T63 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab96670005, negotiated timeout = 20000
[junit4:junit4]   2> 6467 T64 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1020c592 name:ZooKeeperConnection Watcher:127.0.0.1:58321 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6467 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6469 T28 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8cab96670005
[junit4:junit4]   2> 6476 T64 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 6477 T25 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:46382 which had sessionid 0x13d8cab96670005
[junit4:junit4]   2> 6476 T23 oaz.ZooKeeper.close Session: 0x13d8cab96670005 closed
[junit4:junit4]   2> 6478 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 6492 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@68190e6b
[junit4:junit4]   2> 6494 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6494 T65 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 6495 T65 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 6495 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54049
[junit4:junit4]   2> 6496 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54049
[junit4:junit4]   2> 6498 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab96670006 with negotiated timeout 20000 for client /140.211.11.196:54049
[junit4:junit4]   2> 6498 T65 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab96670006, negotiated timeout = 20000
[junit4:junit4]   2> 6499 T66 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68190e6b name:ZooKeeperConnection Watcher:127.0.0.1:58321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6499 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6501 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670006 type:create cxid:0x1 zxid:0x48 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6504 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670006 type:create cxid:0x3 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6508 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7499 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7500 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:32209_pc_kzu%2Fvk_collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32209_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32209/pc_kzu/vk"}
[junit4:junit4]   2> 7513 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 7513 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 7513 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 7514 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32591_pc_kzu%2Fvk
[junit4:junit4]   2> 7515 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670006 type:delete cxid:0xd zxid:0x4e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:32591_pc_kzu%2Fvk Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:32591_pc_kzu%2Fvk
[junit4:junit4]   2> 7517 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32591_pc_kzu%2Fvk
[junit4:junit4]   2> 7527 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 7528 T66 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 7528 T46 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 7529 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7528 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7530 T53 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 7541 T67 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1363864694060/collection1
[junit4:junit4]   2> 7541 T67 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7542 T67 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7543 T67 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7545 T67 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1363864694060/collection1/'
[junit4:junit4]   2> 7546 T67 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1363864694060/collection1/lib/README' to classloader
[junit4:junit4]   2> 7547 T67 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1363864694060/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7604 T67 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 7672 T67 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7774 T67 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7799 T67 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8614 T67 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8635 T67 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8641 T67 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8672 T67 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8678 T67 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8683 T67 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8684 T67 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 8685 T67 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 8685 T67 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8687 T67 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 8687 T67 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 8687 T67 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 8688 T67 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1363864694060/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty1/
[junit4:junit4]   2> 8688 T67 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@636323cc
[junit4:junit4]   2> 8689 T67 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 8690 T67 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty1 forceNew: false
[junit4:junit4]   2> 8690 T67 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty1/index/
[junit4:junit4]   2> 8691 T67 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8691 T67 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty1/index forceNew: false
[junit4:junit4]   2> 8706 T67 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8706 T67 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8709 T67 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8710 T67 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8710 T67 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8711 T67 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8712 T67 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8712 T67 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8713 T67 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8713 T67 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8714 T67 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8723 T67 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8730 T67 oass.SolrIndexSearcher.<init> Opening Searcher@5f61bcbe main
[junit4:junit4]   2> 8731 T67 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8731 T67 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8735 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5f61bcbe main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8738 T67 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8738 T67 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9022 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9024 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32591_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32591/pc_kzu/vk"}
[junit4:junit4]   2> 9024 T47 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 3
[junit4:junit4]   2> 9025 T47 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 9037 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9037 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9037 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9741 T67 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9741 T67 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:32591/pc_kzu/vk collection:collection1 shard:shard1
[junit4:junit4]   2> 9743 T67 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 9763 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670006 type:delete cxid:0x4c zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 9770 T67 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9770 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670006 type:create cxid:0x4d zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9781 T67 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9782 T67 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9782 T67 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:32591/pc_kzu/vk/collection1/
[junit4:junit4]   2> 9782 T67 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9783 T67 oasc.SyncStrategy.syncToMe http://127.0.0.1:32591/pc_kzu/vk/collection1/ has no replicas
[junit4:junit4]   2> 9783 T67 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:32591/pc_kzu/vk/collection1/
[junit4:junit4]   2> 9784 T67 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 9800 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670006 type:create cxid:0x58 zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10544 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10679 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10679 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10679 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10687 T67 oasc.ZkController.register We are http://127.0.0.1:32591/pc_kzu/vk/collection1/ and leader is http://127.0.0.1:32591/pc_kzu/vk/collection1/
[junit4:junit4]   2> 10687 T67 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32591/pc_kzu/vk
[junit4:junit4]   2> 10688 T67 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 10688 T67 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 10688 T67 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10691 T67 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10694 T23 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 10695 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 10696 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10918 T23 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10921 T23 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18856
[junit4:junit4]   2> 10922 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10923 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10924 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1363864698664
[junit4:junit4]   2> 10924 T23 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1363864698664/solr.xml
[junit4:junit4]   2> 10925 T23 oasc.CoreContainer.<init> New CoreContainer 1445412951
[junit4:junit4]   2> 10926 T23 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1363864698664/'
[junit4:junit4]   2> 10926 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1363864698664/'
[junit4:junit4]   2> 11006 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 11007 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11008 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 11008 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11009 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11010 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11010 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11011 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11011 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11012 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 11042 T23 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11067 T23 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58321/solr
[junit4:junit4]   2> 11068 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 11068 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3fd01553
[junit4:junit4]   2> 11070 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11070 T79 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11071 T79 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 11072 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18857
[junit4:junit4]   2> 11073 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18857
[junit4:junit4]   2> 11074 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab96670007 with negotiated timeout 20000 for client /140.211.11.196:18857
[junit4:junit4]   2> 11074 T79 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab96670007, negotiated timeout = 20000
[junit4:junit4]   2> 11075 T80 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3fd01553 name:ZooKeeperConnection Watcher:127.0.0.1:58321 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11076 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11078 T28 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8cab96670007
[junit4:junit4]   2> 11083 T80 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 11083 T23 oaz.ZooKeeper.close Session: 0x13d8cab96670007 closed
[junit4:junit4]   2> 11083 T25 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18857 which had sessionid 0x13d8cab96670007
[junit4:junit4]   2> 11084 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 11099 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@25299a21
[junit4:junit4]   2> 11101 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11101 T81 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11102 T81 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 11102 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18858
[junit4:junit4]   2> 11103 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18858
[junit4:junit4]   2> 11108 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab96670008 with negotiated timeout 20000 for client /140.211.11.196:18858
[junit4:junit4]   2> 11108 T81 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab96670008, negotiated timeout = 20000
[junit4:junit4]   2> 11109 T82 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25299a21 name:ZooKeeperConnection Watcher:127.0.0.1:58321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11109 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11111 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11113 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670008 type:create cxid:0x3 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11123 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12129 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18856_pc_kzu%2Fvk
[junit4:junit4]   2> 12130 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670008 type:delete cxid:0xd zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18856_pc_kzu%2Fvk Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18856_pc_kzu%2Fvk
[junit4:junit4]   2> 12131 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18856_pc_kzu%2Fvk
[junit4:junit4]   2> 12136 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 12136 T66 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 12137 T46 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 12137 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12137 T82 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 12137 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12141 T53 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 12148 T83 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1363864698664/collection1
[junit4:junit4]   2> 12149 T83 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12150 T83 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12150 T83 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12156 T83 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1363864698664/collection1/'
[junit4:junit4]   2> 12157 T83 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1363864698664/collection1/lib/README' to classloader
[junit4:junit4]   2> 12158 T83 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1363864698664/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12193 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12195 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:32591_pc_kzu%2Fvk_collection1",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32591_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32591/pc_kzu/vk"}
[junit4:junit4]   2> 12204 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12204 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12204 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12204 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12221 T83 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 12289 T83 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12390 T83 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12404 T83 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 13109 T83 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13125 T83 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 13130 T83 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 13168 T83 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13174 T83 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13180 T83 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13182 T83 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 13183 T83 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 13183 T83 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13185 T83 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 13185 T83 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 13186 T83 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 13186 T83 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1363864698664/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty2/
[junit4:junit4]   2> 13187 T83 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@636323cc
[junit4:junit4]   2> 13188 T83 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 13189 T83 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty2 forceNew: false
[junit4:junit4]   2> 13190 T83 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty2/index/
[junit4:junit4]   2> 13190 T83 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 13191 T83 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty2/index forceNew: false
[junit4:junit4]   2> 13200 T83 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty2/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13200 T83 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13204 T83 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13204 T83 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13205 T83 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13206 T83 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13207 T83 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13207 T83 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13208 T83 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13209 T83 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13209 T83 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13222 T83 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13230 T83 oass.SolrIndexSearcher.<init> Opening Searcher@5e9a681a main
[junit4:junit4]   2> 13231 T83 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13232 T83 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13238 T84 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5e9a681a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13241 T83 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13242 T83 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13713 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13715 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18856_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18856/pc_kzu/vk"}
[junit4:junit4]   2> 13716 T47 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 13716 T47 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 13729 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13730 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13730 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13730 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14245 T83 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 14245 T83 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:18856/pc_kzu/vk collection:collection1 shard:shard2
[junit4:junit4]   2> 14247 T83 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 14268 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670008 type:delete cxid:0x49 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 14269 T83 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 14270 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670008 type:create cxid:0x4a zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14281 T83 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 14282 T83 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 14282 T83 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:18856/pc_kzu/vk/collection1/
[junit4:junit4]   2> 14282 T83 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 14283 T83 oasc.SyncStrategy.syncToMe http://127.0.0.1:18856/pc_kzu/vk/collection1/ has no replicas
[junit4:junit4]   2> 14283 T83 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:18856/pc_kzu/vk/collection1/
[junit4:junit4]   2> 14284 T83 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 14290 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab96670008 type:create cxid:0x54 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15238 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15267 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 15267 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 15267 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 15267 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 15321 T83 oasc.ZkController.register We are http://127.0.0.1:18856/pc_kzu/vk/collection1/ and leader is http://127.0.0.1:18856/pc_kzu/vk/collection1/
[junit4:junit4]   2> 15321 T83 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18856/pc_kzu/vk
[junit4:junit4]   2> 15322 T83 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 15322 T83 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15322 T83 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15326 T83 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15327 T23 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 15328 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15329 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15529 T23 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15533 T23 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18862
[junit4:junit4]   2> 15534 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15535 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15535 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1363864703291
[junit4:junit4]   2> 15536 T23 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1363864703291/solr.xml
[junit4:junit4]   2> 15536 T23 oasc.CoreContainer.<init> New CoreContainer 1774911181
[junit4:junit4]   2> 15537 T23 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1363864703291/'
[junit4:junit4]   2> 15538 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1363864703291/'
[junit4:junit4]   2> 15607 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 15608 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15609 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15609 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15610 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15610 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15611 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15612 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15612 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15613 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15639 T23 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15661 T23 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58321/solr
[junit4:junit4]   2> 15661 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15662 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@59f74ada
[junit4:junit4]   2> 15663 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15664 T95 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15665 T95 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 15666 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18863
[junit4:junit4]   2> 15666 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18863
[junit4:junit4]   2> 15674 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab96670009 with negotiated timeout 20000 for client /140.211.11.196:18863
[junit4:junit4]   2> 15674 T95 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab96670009, negotiated timeout = 20000
[junit4:junit4]   2> 15675 T96 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59f74ada name:ZooKeeperConnection Watcher:127.0.0.1:58321 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15675 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15678 T28 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8cab96670009
[junit4:junit4]   2> 15686 T96 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15686 T23 oaz.ZooKeeper.close Session: 0x13d8cab96670009 closed
[junit4:junit4]   2> 15686 T25 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18863 which had sessionid 0x13d8cab96670009
[junit4:junit4]   2> 15687 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 15702 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@786ccffd
[junit4:junit4]   2> 15704 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15704 T97 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15705 T97 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 15705 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18864
[junit4:junit4]   2> 15706 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18864
[junit4:junit4]   2> 15710 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab9667000a with negotiated timeout 20000 for client /140.211.11.196:18864
[junit4:junit4]   2> 15710 T97 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab9667000a, negotiated timeout = 20000
[junit4:junit4]   2> 15711 T98 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@786ccffd name:ZooKeeperConnection Watcher:127.0.0.1:58321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15711 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15713 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15729 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000a type:create cxid:0x3 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15733 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 16739 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18862_pc_kzu%2Fvk
[junit4:junit4]   2> 16740 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000a type:delete cxid:0xd zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18862_pc_kzu%2Fvk Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18862_pc_kzu%2Fvk
[junit4:junit4]   2> 16775 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18862_pc_kzu%2Fvk
[junit4:junit4]   2> 16784 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 16785 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16785 T66 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 16785 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16786 T98 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 16786 T82 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 16786 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16787 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:18856_pc_kzu%2Fvk_collection1",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18856_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18856/pc_kzu/vk"}
[junit4:junit4]   2> 16798 T53 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 16805 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16806 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16805 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16805 T98 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16809 T99 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1363864703291/collection1
[junit4:junit4]   2> 16809 T99 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 16815 T99 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 16816 T99 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 16817 T46 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 16817 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16818 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1363864703291/collection1/'
[junit4:junit4]   2> 16820 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1363864703291/collection1/lib/README' to classloader
[junit4:junit4]   2> 16820 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16821 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1363864703291/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 16881 T99 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 16952 T99 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 17053 T99 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 17065 T99 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 17752 T99 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 17764 T99 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 17767 T99 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 17793 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17798 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17802 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17803 T99 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 17804 T99 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 17804 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17806 T99 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 17806 T99 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 17806 T99 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 17807 T99 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1363864703291/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty3/
[junit4:junit4]   2> 17807 T99 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@636323cc
[junit4:junit4]   2> 17808 T99 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 17808 T99 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty3 forceNew: false
[junit4:junit4]   2> 17809 T99 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty3/index/
[junit4:junit4]   2> 17810 T99 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 17811 T99 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty3/index forceNew: false
[junit4:junit4]   2> 17815 T99 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty3/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 17816 T99 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 17818 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 17819 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 17819 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 17820 T99 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 17821 T99 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 17821 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 17821 T99 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 17822 T99 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 17822 T99 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 17833 T99 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 17839 T99 oass.SolrIndexSearcher.<init> Opening Searcher@597a8430 main
[junit4:junit4]   2> 17840 T99 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 17840 T99 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 17844 T100 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@597a8430 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 17847 T99 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 17847 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18318 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18320 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18862_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18862/pc_kzu/vk"}
[junit4:junit4]   2> 18320 T47 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 18320 T47 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4:junit4]   2> 18334 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18334 T98 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18334 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18334 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18334 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18851 T99 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 18851 T99 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:18862/pc_kzu/vk collection:collection1 shard:shard3
[junit4:junit4]   2> 18853 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4:junit4]   2> 18868 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000a type:delete cxid:0x49 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard3 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard3
[junit4:junit4]   2> 18869 T99 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 18870 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000a type:create cxid:0x4a zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18879 T99 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 18880 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 18880 T99 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:18862/pc_kzu/vk/collection1/
[junit4:junit4]   2> 18880 T99 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 18880 T99 oasc.SyncStrategy.syncToMe http://127.0.0.1:18862/pc_kzu/vk/collection1/ has no replicas
[junit4:junit4]   2> 18881 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:18862/pc_kzu/vk/collection1/
[junit4:junit4]   2> 18881 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4:junit4]   2> 18886 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000a type:create cxid:0x54 zxid:0xa3 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 19841 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19864 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19864 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19864 T98 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19864 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19864 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19917 T99 oasc.ZkController.register We are http://127.0.0.1:18862/pc_kzu/vk/collection1/ and leader is http://127.0.0.1:18862/pc_kzu/vk/collection1/
[junit4:junit4]   2> 19917 T99 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18862/pc_kzu/vk
[junit4:junit4]   2> 19917 T99 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 19917 T99 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 19918 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19920 T99 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19922 T23 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 19922 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 19923 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20093 T23 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 20096 T23 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18865
[junit4:junit4]   2> 20097 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 20097 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 20098 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1363864707883
[junit4:junit4]   2> 20098 T23 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1363864707883/solr.xml
[junit4:junit4]   2> 20099 T23 oasc.CoreContainer.<init> New CoreContainer 259111992
[junit4:junit4]   2> 20099 T23 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1363864707883/'
[junit4:junit4]   2> 20100 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1363864707883/'
[junit4:junit4]   2> 20152 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 20153 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 20153 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 20153 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 20154 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 20154 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 20155 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 20155 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 20155 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 20156 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 20171 T23 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 20186 T23 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58321/solr
[junit4:junit4]   2> 20187 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 20187 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@19eac7d5
[junit4:junit4]   2> 20188 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 20189 T111 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 20189 T111 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 20190 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18866
[junit4:junit4]   2> 20190 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18866
[junit4:junit4]   2> 20193 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab9667000b with negotiated timeout 20000 for client /140.211.11.196:18866
[junit4:junit4]   2> 20193 T111 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab9667000b, negotiated timeout = 20000
[junit4:junit4]   2> 20193 T112 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19eac7d5 name:ZooKeeperConnection Watcher:127.0.0.1:58321 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 20193 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 20195 T28 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8cab9667000b
[junit4:junit4]   2> 20207 T112 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 20208 T25 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18866 which had sessionid 0x13d8cab9667000b
[junit4:junit4]   2> 20207 T23 oaz.ZooKeeper.close Session: 0x13d8cab9667000b closed
[junit4:junit4]   2> 20209 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 20217 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5cf901e6
[junit4:junit4]   2> 20218 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 20219 T113 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 20219 T113 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 20220 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18867
[junit4:junit4]   2> 20220 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18867
[junit4:junit4]   2> 20222 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab9667000c with negotiated timeout 20000 for client /140.211.11.196:18867
[junit4:junit4]   2> 20222 T113 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab9667000c, negotiated timeout = 20000
[junit4:junit4]   2> 20222 T114 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cf901e6 name:ZooKeeperConnection Watcher:127.0.0.1:58321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 20223 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 20224 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000c type:create cxid:0x1 zxid:0xb0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 20226 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000c type:create cxid:0x3 zxid:0xb1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 20229 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 21234 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18865_pc_kzu%2Fvk
[junit4:junit4]   2> 21235 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000c type:delete cxid:0xd zxid:0xb2 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18865_pc_kzu%2Fvk Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18865_pc_kzu%2Fvk
[junit4:junit4]   2> 21237 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18865_pc_kzu%2Fvk
[junit4:junit4]   2> 21243 T114 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 21243 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 21243 T66 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 21244 T46 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 21244 T82 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 21243 T98 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 21244 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21244 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21245 T98 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21245 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21248 T53 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 21256 T115 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1363864707883/collection1
[junit4:junit4]   2> 21256 T115 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 21257 T115 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 21257 T115 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 21260 T115 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1363864707883/collection1/'
[junit4:junit4]   2> 21261 T115 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1363864707883/collection1/lib/README' to classloader
[junit4:junit4]   2> 21262 T115 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1363864707883/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 21314 T115 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 21374 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21376 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:18862_pc_kzu%2Fvk_collection1",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard3",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18862_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18862/pc_kzu/vk"}
[junit4:junit4]   2> 21377 T115 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 21389 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21389 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21389 T98 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21389 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21389 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21478 T115 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 21478 T114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21488 T115 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 22204 T115 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 22214 T115 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 22217 T115 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 22242 T115 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 22246 T115 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 22250 T115 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 22252 T115 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 22252 T115 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 22253 T115 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 22254 T115 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 22254 T115 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 22255 T115 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 22255 T115 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1363864707883/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty4/
[junit4:junit4]   2> 22255 T115 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@636323cc
[junit4:junit4]   2> 22256 T115 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 22257 T115 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty4 forceNew: false
[junit4:junit4]   2> 22257 T115 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty4/index/
[junit4:junit4]   2> 22258 T115 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 22258 T115 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty4/index forceNew: false
[junit4:junit4]   2> 22262 T115 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty4/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 22262 T115 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 22265 T115 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 22265 T115 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 22266 T115 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 22267 T115 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 22267 T115 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 22267 T115 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 22268 T115 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 22268 T115 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 22269 T115 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 22277 T115 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 22283 T115 oass.SolrIndexSearcher.<init> Opening Searcher@508892c2 main
[junit4:junit4]   2> 22284 T115 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 22284 T115 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 22289 T116 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@508892c2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 22291 T115 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 22292 T115 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22897 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22899 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18865_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18865/pc_kzu/vk"}
[junit4:junit4]   2> 22899 T47 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 22900 T47 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 22916 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22916 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22916 T114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22916 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22916 T98 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22916 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23294 T115 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 23294 T115 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:18865/pc_kzu/vk collection:collection1 shard:shard1
[junit4:junit4]   2> 23299 T115 oasc.ZkController.register We are http://127.0.0.1:18865/pc_kzu/vk/collection1/ and leader is http://127.0.0.1:32591/pc_kzu/vk/collection1/
[junit4:junit4]   2> 23299 T115 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18865/pc_kzu/vk
[junit4:junit4]   2> 23300 T115 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 23300 T115 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@58456d6e url=http://127.0.0.1:18865/pc_kzu/vk/collection1 node=127.0.0.1:18865_pc_kzu%2Fvk C6_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:18865_pc_kzu%2Fvk, base_url=http://127.0.0.1:18865/pc_kzu/vk}
[junit4:junit4]   2> 23307 T117 C6 P18865 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 23308 T115 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23308 T117 C6 P18865 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 23309 T117 C6 P18865 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 23309 T117 C6 P18865 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 23310 T23 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 23311 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 23312 T117 C6 P18865 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 23312 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 23605 T23 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 23609 T23 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18873
[junit4:junit4]   2> 23610 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 23611 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 23611 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1363864711286
[junit4:junit4]   2> 23612 T23 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1363864711286/solr.xml
[junit4:junit4]   2> 23613 T23 oasc.CoreContainer.<init> New CoreContainer 2144609923
[junit4:junit4]   2> 23613 T23 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1363864711286/'
[junit4:junit4]   2> 23614 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1363864711286/'
[junit4:junit4]   2> 23709 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 23710 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 23711 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 23711 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 23712 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 23712 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 23713 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 23714 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 23714 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 23715 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 23745 T23 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 23770 T23 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58321/solr
[junit4:junit4]   2> 23771 T23 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 23771 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6095cc5b
[junit4:junit4]   2> 23773 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 23773 T128 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 23774 T128 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 23774 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18875
[junit4:junit4]   2> 23775 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18875
[junit4:junit4]   2> 23777 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab9667000d with negotiated timeout 20000 for client /140.211.11.196:18875
[junit4:junit4]   2> 23777 T128 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab9667000d, negotiated timeout = 20000
[junit4:junit4]   2> 23778 T129 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6095cc5b name:ZooKeeperConnection Watcher:127.0.0.1:58321 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 23778 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 23780 T28 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8cab9667000d
[junit4:junit4]   2> 23782 T129 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 23782 T23 oaz.ZooKeeper.close Session: 0x13d8cab9667000d closed
[junit4:junit4]   2> 23782 T25 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18875 which had sessionid 0x13d8cab9667000d
[junit4:junit4]   2> 23783 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 23797 T23 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58321/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@2bf525ec
[junit4:junit4]   2> 23799 T23 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 23799 T130 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58321. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 23800 T130 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58321, initiating session
[junit4:junit4]   2> 23800 T25 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18876
[junit4:junit4]   2> 23801 T25 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18876
[junit4:junit4]   2> 23802 T27 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8cab9667000e with negotiated timeout 20000 for client /140.211.11.196:18876
[junit4:junit4]   2> 23802 T130 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58321, sessionid = 0x13d8cab9667000e, negotiated timeout = 20000
[junit4:junit4]   2> 23803 T131 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2bf525ec name:ZooKeeperConnection Watcher:127.0.0.1:58321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 23803 T23 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 23805 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000e type:create cxid:0x1 zxid:0xc3 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 23808 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000e type:create cxid:0x3 zxid:0xc4 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 23812 T23 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 24424 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 24426 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:18865_pc_kzu%2Fvk_collection1",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18865_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18865/pc_kzu/vk"}
[junit4:junit4]   2> 24435 T114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24435 T131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24435 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24435 T98 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24435 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24435 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24435 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24570 T56 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:18865_pc_kzu%252Fvk_collection1&state=recovering&nodeName=127.0.0.1:18865_pc_kzu%252Fvk&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=4 
[junit4:junit4]   2> 24817 T23 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18873_pc_kzu%2Fvk
[junit4:junit4]   2> 24819 T28 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8cab9667000e type:delete cxid:0xf zxid:0xc9 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18873_pc_kzu%2Fvk Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18873_pc_kzu%2Fvk
[junit4:junit4]   2> 24821 T23 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18873_pc_kzu%2Fvk
[junit4:junit4]   2> 24824 T114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24824 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24824 T131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24825 T82 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 24825 T46 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 24825 T66 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 24825 T98 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 24827 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 24827 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 24826 T114 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 24828 T98 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 24828 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 24834 T53 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 24835 T131 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 24844 T132 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1363864711286/collection1
[junit4:junit4]   2> 24844 T132 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 24845 T132 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 24846 T132 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 24848 T132 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1363864711286/collection1/'
[junit4:junit4]   2> 24849 T132 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1363864711286/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 24850 T132 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1363864711286/collection1/lib/README' to classloader
[junit4:junit4]   2> 24911 T132 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 24961 T132 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 24962 T132 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 24969 T132 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 25459 T132 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 25468 T132 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 25483 T132 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 25507 T132 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 25511 T132 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 25515 T132 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 25517 T132 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 25517 T132 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 25517 T132 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 25519 T132 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 25519 T132 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 25519 T132 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 25520 T132 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1363864711286/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty5/
[junit4:junit4]   2> 25520 T132 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@636323cc
[junit4:junit4]   2> 25520 T132 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 25521 T132 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty5 forceNew: false
[junit4:junit4]   2> 25522 T132 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty5/index/
[junit4:junit4]   2> 25522 T132 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 25523 T132 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty5/index forceNew: false
[junit4:junit4]   2> 25548 T132 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty5/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 25548 T132 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 25551 T132 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 25551 T132 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 25552 T132 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 25553 T132 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 25553 T132 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 25553 T132 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 25554 T132 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 25555 T132 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 25555 T132 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 25564 T132 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 25569 T132 oass.SolrIndexSearcher.<init> Opening Searcher@660dec8b main
[junit4:junit4]   2> 25570 T132 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 25571 T132 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 25576 T133 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@660dec8b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 25578 T132 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 25578 T132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 25949 T47 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 25951 T47 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18873_pc_kzu%2Fvk",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18873/pc_kzu/vk"}
[junit4:junit4]   2> 25951 T47 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 25951 T47 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 25960 T114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 25960 T82 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 25960 T98 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 25960 T131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 25960 T66 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 25960 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 25960 T46 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 26617 T132 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 26617 T132 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:18873/pc_kzu/vk collection:collection1 shard:shard2
[junit4:junit4]   2> 26623 T132 oasc.ZkController.register We are http://127.0.0.1:18873/pc_kzu/vk/collection1/ and leader is http://127.0.0.1:18856/pc_kzu/vk/collection1/
[junit4:junit4]   2> 26623 T132 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18873/pc_kzu/vk
[junit4:junit4]   2> 26623 T132 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 26624 T132 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C7 name=collection1 org.apache.solr.core.SolrCore@4062c054 url=http://127.0.0.1:18873/pc_kzu/vk/collection1 node=127.0.0.1:18873_pc_kzu%2Fvk C7_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:18873_pc_kzu%2Fvk, base_url=http://127.0.0.1:18873/pc_kzu/vk}
[junit4:junit4]   2> 26624 T134 C7 P18873 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 26625 T134 C7 P18873 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 26625 T132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 26625 T134 C7 P18873 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 26626 T134 C7 P18873 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 26627 T23 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 26628 T134 C7 P18873 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 26628 T23 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 26629 T23 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2>  C6_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:18865_pc_kzu%2Fvk, base_url=http://127.0.0.1:18865/pc_kzu/vk}
[junit4:junit4]   2> 26703 T117 C6 P18865 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:32591/pc_kzu/vk/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 26715 T117 C6 P18865 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 26737 T117 C6 P18865 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:18865/pc_kzu/vk START replicas=[http://127.0.0.1:32591/pc_kzu/vk/collection1/] nUpdates=100
[junit4:junit4]   2> 26751 T117 C6 P18865 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 26751 T117 C6 P18865 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 26752 T117 C6 P18865 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 26752 T117 C6 P18865 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 26753 T117 C6 P18865 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 26753 T117 C6 P18865 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:32591/pc_kzu/vk/collection1/. core=collection1
[junit4:junit4]   2> 26753 T117 C6 P18865 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C8 name=collection1 org.apache.solr.core.SolrCore@2c3e866 url=http://127.0.0.1:32591/pc_kzu/vk/collection1 node=127.0.0.1:32591_pc_kzu%2Fvk C8_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:32591_pc_kzu%2Fvk, base_url=http://127.0.0.1:32591/pc_kzu/vk, leader=true}
[junit4:junit4]   2> 26813 T58 C8 P32591 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=7 
[junit4:junit4]   2> 26818 T57 C8 P32591 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 26824 T57 C8 P32591 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1363864687968/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 26825 T57 C8 P32591 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 26845 T57 C8 P32591 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.

[...truncated too long message...]

dsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 145245 T1739 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 173815 T1833 oasu.PeerSync.handleResponse WARNING PeerSync: core=collection1 url=http://127.0.0.1:44927/b/rt  couldn't connect to http://127.0.0.1:44920/b/rt/collection1/, counting as success
[junit4:junit4]   2> 173816 T1833 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44927/b/rt DONE. sync succeeded
[junit4:junit4]   2> 173816 T1833 oasc.SyncStrategy.syncReplicas We have been closed, won't attempt to sync replicas back to leader
[junit4:junit4]   2> 173818 T1833 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
[junit4:junit4]   2> 173818 T1833 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1da63a80
[junit4:junit4]   2> 173819 T1833 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=10,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=305,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 173820 T1833 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 173820 T1833 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 173820 T1833 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 173821 T1833 C100 P44927 oasu.TransactionLog.forceClose SEVERE Error: Forcing close of tlog{file=./org.apache.solr.cloud.SyncSliceTest-1363865057797/jetty4/tlog/tlog.0000000000000000005 refcount=1}
[junit4:junit4]   2> 173823 T1833 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 173825 T1833 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363865057797/jetty4
[junit4:junit4]   2> 173825 T1833 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363865057797/jetty4
[junit4:junit4]   2> 173825 T1833 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363865057797/jetty4/index
[junit4:junit4]   2> 173826 T1833 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363865057797/jetty4/index
[junit4:junit4]   2> 173826 T1833 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363865057797/jetty4/index.20130321072442869
[junit4:junit4]   2> 173826 T1833 oasc.CachingDirectoryFactory.closeDirectory Removing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363865057797/jetty4/index.20130321072442869
[junit4:junit4]   2> 173827 T1833 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363865057797/jetty4/index.20130321072442869
[junit4:junit4]   2> 173827 T1833 oasc.LeaderElector$1.process WARNING  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> 173828 T1833 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 173828 T1833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 173828 T1833 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 173829 T1833 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=ja_JP, timezone=America/Toronto
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_17 (64-bit)/cpus=16,threads=3,free=248975376,total=476053504
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestStressLucene, TestLRUCache, StandardRequestHandlerTest, TestAnalyzedSuggestions, DocValuesMultiTest, ScriptEngineTest, PingRequestHandlerTest, LegacyHTMLStripCharFilterTest, TestSort, DisMaxRequestHandlerTest, RegexBoostProcessorTest, UpdateRequestProcessorFactoryTest, AlternateDirectoryTest, ZkCLITest, TestCharFilters, CoreAdminHandlerTest, QueryParsingTest, SolrCoreCheckLockOnStartupTest, TestBM25SimilarityFactory, OutputWriterTest, TestElisionMultitermQuery, OverseerTest, SliceStateUpdateTest, RecoveryZkTest, TestRecovery, TestShardHandlerFactory, TestQuerySenderNoQuery, TestFiltering, SystemInfoHandlerTest, TestLMDirichletSimilarityFactory, UniqFieldsUpdateProcessorFactoryTest, URLClassifyProcessorTest, TestCollationKeyRangeQueries, TestIndexSearcher, TestDynamicFieldCollectionResource, WordBreakSolrSpellCheckerTest, TestWordDelimiterFilterFactory, BinaryUpdateRequestHandlerTest, SimplePostToolTest, SchemaVersionSpecificBehaviorTest, SimpleFacetsTest, TestIndexingPerformance, TestFastOutputStream, AutoCommitTest, XmlUpdateRequestHandlerTest, TestPartialUpdateDeduplication, QueryElevationComponentTest, TestFastWriter, TestSolrDeletionPolicy1, FileUtilsTest, TestIBSimilarityFactory, TestCSVResponseWriter, TestJmxMonitoredMap, SortByFunctionTest, DistributedSpellCheckComponentTest, OverseerCollectionProcessorTest, TestSystemIdResolver, BadIndexSchemaTest, MoreLikeThisHandlerTest, DebugComponentTest, TestUpdate, TestReplicationHandler, NoCacheHeaderTest, SynonymTokenizerTest, SolrRequestParserTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 174.31s, 1 test, 1 failure <<< FAILURES!

[...truncated 479 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:877: There were test failures: 273 suites, 1147 tests, 2 failures, 27 ignored (8 assumptions)

Total time: 47 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message