lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3301 - Failure
Date Sun, 14 Oct 2012 02:40:47 GMT
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3301/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.RecoveryZkTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:22439/solr/collection1 returned non ok status:404, message:Can not find: /solr/collection1/select

Stack Trace:
org.apache.solr.common.SolrException: Server at http://127.0.0.1:22439/solr/collection1 returned non ok status:404, message:Can not find: /solr/collection1/select
	at __randomizedtesting.SeedInfo.seed([E26F3CBC79627AFE:6389B2A40E3D1AC2]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:372)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
	at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:92)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:695)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 8426 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4]   2> 12 T17 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-RecoveryZkTest-1350181602298
[junit4:junit4]   2> 24 T17 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 137 T18 oaz.Environment.logEnv Server environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 138 T18 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 138 T18 oaz.Environment.logEnv Server environment:java.version=1.7.0_06
[junit4:junit4]   2> 150 T18 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 150 T18 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 150 T18 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.3.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.3.jar
[junit4:junit4]   2> 152 T18 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 152 T18 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 152 T18 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 153 T18 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 153 T18 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 153 T18 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 153 T18 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 154 T18 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 154 T18 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 194 T18 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 226 T18 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 234 T17 oasc.ZkTestServer.run start zk server on port:22394
[junit4:junit4]   2> 312 T18 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 343 T17 oaz.Environment.logEnv Client environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 344 T17 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 345 T17 oaz.Environment.logEnv Client environment:java.version=1.7.0_06
[junit4:junit4]   2> 345 T17 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 346 T17 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 346 T17 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.3.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.3.jar
[junit4:junit4]   2> 347 T17 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 348 T17 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 348 T17 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 349 T17 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 349 T17 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 350 T17 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 350 T17 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 350 T17 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 351 T17 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 358 T17 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22394 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7fb2832e
[junit4:junit4]   2> 428 T22 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:22394
[junit4:junit4]   2> 428 T17 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 431 T22 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22394, initiating session
[junit4:junit4]   2> 432 T19 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:22396
[junit4:junit4]   2> 450 T19 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4]   2> 450 T19 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22396 (no session established for client)
[junit4:junit4]   2> 451 T22 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 2397 T22 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost/127.0.0.1:22394
[junit4:junit4]   2> 2398 T22 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22394, initiating session
[junit4:junit4]   2> 2398 T19 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:22401
[junit4:junit4]   2> 2400 T19 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:22401
[junit4:junit4]   2> 2421 T21 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 2503 T21 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a5d1845510000 with negotiated timeout 10000 for client /140.211.11.196:22401
[junit4:junit4]   2> 2503 T22 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:22394, sessionid = 0x13a5d1845510000, negotiated timeout = 10000
[junit4:junit4]   2> 2510 T24 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7fb2832e name:ZooKeeperConnection Watcher:127.0.0.1:22394 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2510 T17 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2514 T17 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2609 T23 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13a5d1845510000
[junit4:junit4]   2> 2611 T24 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2611 T17 oaz.ZooKeeper.close Session: 0x13a5d1845510000 closed
[junit4:junit4]   2> 2612 T17 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22394/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@aa1aeea
[junit4:junit4]   2> 2612 T19 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22401 which had sessionid 0x13a5d1845510000
[junit4:junit4]   2> 2613 T25 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:22394
[junit4:junit4]   2> 2613 T17 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2614 T25 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22394, initiating session
[junit4:junit4]   2> 2614 T19 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:22402
[junit4:junit4]   2> 2615 T19 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:22402
[junit4:junit4]   2> 2617 T21 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a5d1845510001 with negotiated timeout 10000 for client /140.211.11.196:22402
[junit4:junit4]   2> 2617 T25 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:22394, sessionid = 0x13a5d1845510001, negotiated timeout = 10000
[junit4:junit4]   2> 2618 T26 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@aa1aeea name:ZooKeeperConnection Watcher:127.0.0.1:22394/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2619 T17 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2639 T17 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2646 T17 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2650 T17 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2654 T17 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2659 T17 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2683 T17 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2699 T17 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2700 T17 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2831 T17 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2832 T17 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2875 T17 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2876 T17 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2880 T17 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2881 T17 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2885 T17 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2886 T17 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2890 T17 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2891 T17 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2896 T17 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2897 T17 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2901 T17 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2902 T17 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2906 T17 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2907 T17 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2911 T23 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13a5d1845510001
[junit4:junit4]   2> 2913 T26 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2913 T19 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22402 which had sessionid 0x13a5d1845510001
[junit4:junit4]   2> 2913 T17 oaz.ZooKeeper.close Session: 0x13a5d1845510001 closed
[junit4:junit4]   2> 3480 T17 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3734 T17 oejs.AbstractConnector.doStart Started SocketConnector@0.0.0.0:22410
[junit4:junit4]   2> 3764 T17 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3765 T17 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199
[junit4:junit4]   2> 3766 T17 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199/'
[junit4:junit4]   2> 3803 T17 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3803 T17 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3804 T17 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199
[junit4:junit4]   2> 3805 T17 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199/solr.xml
[junit4:junit4]   2> 3805 T17 oasc.CoreContainer.<init> New CoreContainer 432819206
[junit4:junit4]   2> 3806 T17 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199/'
[junit4:junit4]   2> 3807 T17 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199/'
[junit4:junit4]   2> 3847 T17 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3862 T17 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:22394/solr
[junit4:junit4]   2> 3895 T17 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22394/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@1f673ee6
[junit4:junit4]   2> 3896 T37 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:22394
[junit4:junit4]   2> 3896 T17 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3898 T37 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22394, initiating session
[junit4:junit4]   2> 3898 T19 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:22412
[junit4:junit4]   2> 3899 T19 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:22412
[junit4:junit4]   2> 3901 T21 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a5d1845510002 with negotiated timeout 5000 for client /140.211.11.196:22412
[junit4:junit4]   2> 3901 T37 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:22394, sessionid = 0x13a5d1845510002, negotiated timeout = 5000
[junit4:junit4]   2> 3902 T38 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f673ee6 name:ZooKeeperConnection Watcher:127.0.0.1:22394/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3902 T17 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3920 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3931 T17 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 3934 T17 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22410_solr
[junit4:junit4]   2> 3939 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22410_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22410_solr
[junit4:junit4]   2> 3941 T17 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22410_solr
[junit4:junit4]   2> 3945 T17 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3945 T17 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3946 T17 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3946 T17 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3947 T17 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3947 T17 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3948 T17 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3948 T17 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3949 T17 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3949 T17 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3964 T17 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 3989 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 3991 T17 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 3995 T17 oasc.Overseer.start Overseer (id=88485501510680578-127.0.0.1:22410_solr-n_0000000000) starting
[junit4:junit4]   2> 3996 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4009 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4011 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4018 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4020 T40 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 4021 T17 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 4027 T17 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4047 T39 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 4082 T17 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199/collection1
[junit4:junit4]   2> 4083 T17 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 4084 T17 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4085 T17 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 4087 T17 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199/collection1/'
[junit4:junit4]   2> 4088 T17 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4089 T17 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199/collection1/lib/README' to classloader
[junit4:junit4]   2> 4232 T17 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4367 T17 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4469 T17 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4513 T17 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5082 T17 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5096 T17 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5100 T17 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5125 T17 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5131 T17 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5136 T17 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5140 T17 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5144 T17 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1350181605199/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1350181602297/control/data/
[junit4:junit4]   2> 5152 T17 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@399342e7
[junit4:junit4]   2> 5203 T17 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1350181602297/control/data/index/
[junit4:junit4]   2> 5205 T17 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1350181602297/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5206 T17 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1350181602297/control/data/index forceNew:false
[junit4:junit4]   2> 5238 T17 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1350181602297/control/data/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5239 T17 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5267 T17 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5384 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5385 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5386 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5392 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5481 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5482 T17 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5486 T17 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5487 T17 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5503 T17 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5550 T17 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5553 T17 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5557 T17 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5558 T17 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5559 T17 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5559 T17 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5573 T17 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5574 T17 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5575 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5576 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5577 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5578 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5580 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5581 T17 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5598 T17 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5615 T17 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5624 T17 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5629 T17 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5635 T17 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5639 T17 oass.SolrIndexSearcher.<init> Opening Searcher@173852ee main
[junit4:junit4]   2> 5664 T17 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5665 T17 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5666 T17 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5750 T17 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5957 T41 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@173852ee main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5969 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:create cxid:0x59 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 5982 T17 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 5983 T17 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:22410/solr shardId:control_shard
[junit4:junit4]   2> 5995 T17 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 6008 T17 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/control_shard/election
[junit4:junit4]   2> 6026 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:delete cxid:0x68 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 6028 T17 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6029 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:create cxid:0x69 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6058 T39 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6069 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:create cxid:0x70 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 6105 T38 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 6539 T17 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6540 T17 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6540 T17 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:22410/solr/collection1/
[junit4:junit4]   2> 6541 T17 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6542 T17 oasc.SyncStrategy.syncToMe http://127.0.0.1:22410/solr/collection1/ has no replicas
[junit4:junit4]   2> 6543 T17 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:22410/solr/collection1/
[junit4:junit4]   2> 6544 T17 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/control_shard
[junit4:junit4]   2> 6551 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510002 type:create cxid:0x8e zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6614 T39 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6651 T38 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 6676 T17 oasc.ZkController.register We are http://127.0.0.1:22410/solr/collection1/ and leader is http://127.0.0.1:22410/solr/collection1/
[junit4:junit4]   2> 6681 T17 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22410/solr
[junit4:junit4]   2> 6682 T17 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6685 T17 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6687 T17 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 6687 T17 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6768 T17 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 6924 T17 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 6927 T17 oejs.AbstractConnector.doStart Started SocketConnector@0.0.0.0:22421
[junit4:junit4]   2> 6928 T17 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6928 T17 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078
[junit4:junit4]   2> 6929 T17 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078/'
[junit4:junit4]   2> 6961 T17 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 6962 T17 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6962 T17 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078
[junit4:junit4]   2> 6963 T17 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078/solr.xml
[junit4:junit4]   2> 6963 T17 oasc.CoreContainer.<init> New CoreContainer 863145246
[junit4:junit4]   2> 6964 T17 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078/'
[junit4:junit4]   2> 6964 T17 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078/'
[junit4:junit4]   2> 6990 T17 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7012 T17 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:22394/solr
[junit4:junit4]   2> 7013 T17 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22394/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@69da79bb
[junit4:junit4]   2> 7014 T51 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:22394
[junit4:junit4]   2> 7014 T17 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7015 T51 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22394, initiating session
[junit4:junit4]   2> 7015 T19 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:22422
[junit4:junit4]   2> 7016 T19 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:22422
[junit4:junit4]   2> 7021 T21 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a5d1845510003 with negotiated timeout 5000 for client /140.211.11.196:22422
[junit4:junit4]   2> 7021 T51 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:22394, sessionid = 0x13a5d1845510003, negotiated timeout = 5000
[junit4:junit4]   2> 7022 T52 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69da79bb name:ZooKeeperConnection Watcher:127.0.0.1:22394/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7023 T17 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7024 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7037 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7039 T17 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22421_solr
[junit4:junit4]   2> 7040 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22421_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22421_solr
[junit4:junit4]   2> 7041 T17 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22421_solr
[junit4:junit4]   2> 7049 T38 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 7050 T17 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7051 T17 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7052 T17 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7053 T17 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7055 T17 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7056 T17 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7057 T17 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7058 T17 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7059 T17 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7060 T17 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 7090 T17 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7099 T17 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078/collection1
[junit4:junit4]   2> 7099 T17 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7101 T17 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7101 T17 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7104 T17 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078/collection1/'
[junit4:junit4]   2> 7106 T17 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7107 T17 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078/collection1/lib/README' to classloader
[junit4:junit4]   2> 7159 T39 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7175 T52 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 7175 T38 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 7208 T17 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7303 T17 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7405 T17 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7466 T17 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8233 T17 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8246 T17 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8256 T17 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8285 T17 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8291 T17 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8296 T17 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8300 T17 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8303 T17 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1350181609078/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty1/
[junit4:junit4]   2> 8304 T17 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@399342e7
[junit4:junit4]   2> 8305 T17 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty1/index/
[junit4:junit4]   2> 8306 T17 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8307 T17 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty1/index forceNew:false
[junit4:junit4]   2> 8315 T17 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8316 T17 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 8317 T17 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8320 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8320 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8321 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8322 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8323 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8323 T17 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8324 T17 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8325 T17 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8326 T17 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8328 T17 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8329 T17 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8330 T17 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8331 T17 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8331 T17 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8332 T17 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8334 T17 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8335 T17 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8336 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8337 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8338 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8339 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8341 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8342 T17 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8343 T17 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8344 T17 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8353 T17 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8357 T17 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8359 T17 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8363 T17 oass.SolrIndexSearcher.<init> Opening Searcher@7914bbd2 main
[junit4:junit4]   2> 8364 T17 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8365 T17 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8366 T17 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8379 T17 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8442 T53 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7914bbd2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8689 T39 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8706 T52 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 8706 T38 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 8956 T17 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 8956 T17 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:22421/solr shardId:shard1
[junit4:junit4]   2> 8958 T17 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 8977 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510003 type:delete cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 8981 T17 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 8982 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510003 type:create cxid:0x44 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8992 T17 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 8992 T17 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 8993 T17 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:22421/solr/collection1/
[junit4:junit4]   2> 8994 T17 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 8995 T17 oasc.SyncStrategy.syncToMe http://127.0.0.1:22421/solr/collection1/ has no replicas
[junit4:junit4]   2> 8995 T17 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:22421/solr/collection1/
[junit4:junit4]   2> 8996 T17 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 9003 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510003 type:create cxid:0x4e zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9212 T39 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9234 T52 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 9234 T38 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 9268 T17 oasc.ZkController.register We are http://127.0.0.1:22421/solr/collection1/ and leader is http://127.0.0.1:22421/solr/collection1/
[junit4:junit4]   2> 9269 T17 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22421/solr
[junit4:junit4]   2> 9269 T17 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9273 T17 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9275 T17 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 9275 T17 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9276 T17 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9424 T17 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 9427 T17 oejs.AbstractConnector.doStart Started SocketConnector@0.0.0.0:22439
[junit4:junit4]   2> 9428 T17 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9429 T17 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574
[junit4:junit4]   2> 9429 T17 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574/'
[junit4:junit4]   2> 9464 T17 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9465 T17 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9466 T17 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574
[junit4:junit4]   2> 9466 T17 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574/solr.xml
[junit4:junit4]   2> 9467 T17 oasc.CoreContainer.<init> New CoreContainer 367793955
[junit4:junit4]   2> 9467 T17 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574/'
[junit4:junit4]   2> 9468 T17 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574/'
[junit4:junit4]   2> 9499 T17 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9514 T17 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:22394/solr
[junit4:junit4]   2> 9515 T17 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22394/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@916a966
[junit4:junit4]   2> 9516 T63 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:22394
[junit4:junit4]   2> 9516 T17 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9518 T63 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22394, initiating session
[junit4:junit4]   2> 9518 T19 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:22440
[junit4:junit4]   2> 9519 T19 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:22440
[junit4:junit4]   2> 9522 T21 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a5d1845510004 with negotiated timeout 5000 for client /140.211.11.196:22440
[junit4:junit4]   2> 9522 T63 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:22394, sessionid = 0x13a5d1845510004, negotiated timeout = 5000
[junit4:junit4]   2> 9523 T64 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@916a966 name:ZooKeeperConnection Watcher:127.0.0.1:22394/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9524 T17 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9525 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9538 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9541 T17 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22439_solr
[junit4:junit4]   2> 9542 T23 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13a5d1845510004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22439_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22439_solr
[junit4:junit4]   2> 9544 T17 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22439_solr
[junit4:junit4]   2> 9547 T38 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 9547 T52 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 9549 T17 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 9550 T17 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9550 T17 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 9551 T17 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9551 T17 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9552 T17 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9552 T17 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9553 T17 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9553 T17 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9554 T17 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 9575 T17 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9583 T17 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574/collection1
[junit4:junit4]   2> 9584 T17 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9586 T17 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9586 T17 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9589 T17 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574/collection1/'
[junit4:junit4]   2> 9590 T17 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574/collection1/lib/README' to classloader
[junit4:junit4]   2> 9591 T17 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9657 T17 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 9729 T17 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9747 T39 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9761 T38 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 9761 T52 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 9830 T64 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 9830 T17 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9867 T17 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10674 T17 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10694 T17 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10701 T17 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10737 T17 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10746 T17 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10754 T17 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10758 T17 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10762 T17 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1350181611574/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty2/
[junit4:junit4]   2> 10763 T17 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@399342e7
[junit4:junit4]   2> 10765 T17 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty2/index/
[junit4:junit4]   2> 10766 T17 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10767 T17 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty2/index forceNew:false
[junit4:junit4]   2> 10776 T17 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty2/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10777 T17 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10778 T17 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10782 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10783 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10784 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10786 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10787 T17 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10788 T17 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10789 T17 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10790 T17 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10792 T17 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10794 T17 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10795 T17 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 10796 T17 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10798 T17 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 10799 T17 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 10800 T17 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 10801 T17 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10803 T17 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10805 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10806 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10808 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10810 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10812 T17 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10814 T17 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10817 T17 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 10824 T17 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 10844 T17 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10849 T17 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 10852 T17 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 10858 T17 oass.SolrIndexSearcher.<init> Opening Searcher@75fde673 main
[junit4:junit4]   2> 10860 T17 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10860 T17 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10862 T17 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 10871 T17 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 10934 T65 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75fde673 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11279 T39 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11343 T64 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 11343 T52 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 11343 T38 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 11459 T17 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 11460 T17 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:22439/solr shardId:shard1
[junit4:junit4]   2> 11477 T17 oasc.ZkController.register We are http://127.0.0.1:22439/solr/collection1/ and leader is http://127.0.0.1:22421/solr/collection1/
[junit4:junit4]   2> 11478 T17 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22439/solr
[junit4:junit4]   2> 11478 T17 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 11479 T17 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@3c6394e6 url=http://127.0.0.1:22439/solr/collection1 node=127.0.0.1:22439_solr
[junit4:junit4]   2> 11488 T66 C2 P22439 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 11489 T17 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11489 T66 C2 P22439 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 11490 T17 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 11491 T17 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11491 T17 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11496 T66 C2 P22439 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:22421/solr/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 11505 T66 C2 P22439 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 11511 T17 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22394/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7e2267cd
[junit4:junit4]   2> 11512 T67 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:22394
[junit4:junit4]   2> 11512 T17 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11513 T67 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22394, initiating session
[junit4:junit4]   2> 11513 T19 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:22445
[junit4:junit4]   2> 11514 T19 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:22445
[junit4:junit4]   2> 11520 T66 C2 P22439 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:22439/solr START replicas=[http://127.0.0.1:22421/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 11532 T66 C2 P22439 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 11532 T66 C2 P22439 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 11533 T66 C2 P22439 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 11533 T66 C2 P22439 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11552 T21 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a5d1845510005 with negotiated timeout 10000 for client /140.211.11.196:22445
[junit4:junit4]   2> 11552 T67 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:22394, sessionid = 0x13a5d1845510005, negotiated timeout = 10000
[junit4:junit4]   2> 11553 T68 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e2267cd name:ZooKeeperConnection Watcher:127.0.0.1:22394/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11553 T17 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11554 T17 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 11559 T17 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 11560 T17 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 11573 T17 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11851 T39 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11869 T64 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 11869 T68 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 11869 T52 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> 11869 T38 oascc.ZkStateReader$2.process A cluster state change has occurred - updating...
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@6d2c0f26 url=http://127.0.0.1:22421/solr/collection1 node=127.0.0.1:22421_solr
[junit4:junit4]   2> 12531 T44 C3 P22421 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=30 
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@733c783d url=http://127.0.0.1:22410/solr/collection1 node=127.0.0.1:22410_solr
[junit4:junit4]   2> 12595 T32 C4 P22410 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16
[junit4:junit4]   2> 12646 T32 C4 P22410 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1350181602297/control/data/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12646 T32 C4 P22410 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 12672 T32 C4 P22410 /update {wt=javabin&version=2} {add=[1 (1415768037051596800)]} 0 128
[junit4:junit4]   2> 12672 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10001 (1415768037052645376)]} 0 128
[junit4:junit4]   2> 12673 T70 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22394/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@289c0574
[junit4:junit4]   2> 12675 T72 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:22394
[junit4:junit4]   2> 12675 T70 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12676 T72 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22394, initiating session
[junit4:junit4]   2> 12677 T19 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:22451
[junit4:junit4]   2> 12677 T19 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:22451
[junit4:junit4]   2> 12679 T21 oazs.NIOServerCnxn.finishSessionInit Established session 0x13a5d1845510006 with negotiated timeout 10000 for client /140.211.11.196:22451
[junit4:junit4]   2> 12679 T72 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:22394, sessionid = 0x13a5d1845510006, negotiated timeout = 10000
[junit4:junit4]   2> 12680 T73 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@289c0574 name:ZooKeeperConnection Watcher:127.0.0.1:22394/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12680 T70 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12682 T70 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12707 T46 C3 P22421 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12708 T46 C3 P22421 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2>  C2_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:22439_solr, base_url=http://127.0.0.1:22439/solr}
[junit4:junit4]   2> 12759 T57 C2 P22439 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty2/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12760 T57 C2 P22439 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 12765 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=11 
[junit4:junit4]   2> 12767 T47 C3 P22421 /update {wt=javabin&version=2} {add=[1 (1415768037116608513)]} 0 71
[junit4:junit4]   2> 12769 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=12 
[junit4:junit4]   2> 12771 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10001 (1415768037116608512)]} 0 74
[junit4:junit4]   2> 12773 T29 C4 P22410 /update {wt=javabin&version=2} {add=[2 (1415768037187911680)]} 0 2
[junit4:junit4]   2> 12778 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10002 (1415768037192105984)]} 0 3
[junit4:junit4]   2> 12788 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 12790 T46 C3 P22421 /update {wt=javabin&version=2} {add=[2 (1415768037195251712)]} 0 12
[junit4:junit4]   2> 12795 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 12796 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10002 (1415768037199446016)]} 0 14
[junit4:junit4]   2> 12799 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[1 (-1415768037217271808)]} 0 5
[junit4:junit4]   2> 12802 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10001 (-1415768037219368960)]} 0 1
[junit4:junit4]   2> 12812 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 12812 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 12813 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10001 (-1415768037224611840)]} 0 8
[junit4:junit4]   2> 12813 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[1 (-1415768037222514688)]} 0 9
[junit4:junit4]   2> 12820 T29 C4 P22410 /update {wt=javabin&version=2} {add=[3 (1415768037237194752)]} 0 3
[junit4:junit4]   2> 12823 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10003 (1415768037237194753)]} 0 5
[junit4:junit4]   2> 12832 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 12833 T46 C3 P22421 /update {wt=javabin&version=2} {add=[3 (1415768037243486208)]} 0 9
[junit4:junit4]   2> 12835 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 12836 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10003 (1415768037246631936)]} 0 10
[junit4:junit4]   2> 12837 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[3 (-1415768037257117696)]} 0 0
[junit4:junit4]   2> 12840 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10003 (-1415768037260263424)]} 0 0
[junit4:junit4]   2> 12845 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 12846 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[3 (-1415768037262360576)]} 0 5
[junit4:junit4]   2> 12848 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 12850 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10003 (-1415768037264457728)]} 0 6
[junit4:junit4]   2> 12854 T29 C4 P22410 /update {wt=javabin&version=2} {add=[4 (1415768037271797760)]} 0 3
[junit4:junit4]   2> 12857 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10004 (1415768037274943488)]} 0 4
[junit4:junit4]   2> 12870 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 12871 T46 C3 P22421 /update {wt=javabin&version=2} {add=[4 (1415768037280186368)]} 0 13
[junit4:junit4]   2> 12872 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 12873 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10004 (1415768037283332096)]} 0 11
[junit4:junit4]   2> 12879 T32 C4 P22410 /update {wt=javabin&version=2} {add=[5 (1415768037296963584)]} 0 4
[junit4:junit4]   2> 12892 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 12893 T47 C3 P22421 /update {wt=javabin&version=2} {add=[5 (1415768037305352192)]} 0 11
[junit4:junit4]   2> 12905 T32 C4 P22410 /update {wt=javabin&version=2} {add=[6 (1415768037321080832)]} 0 7
[junit4:junit4]   2> 12953 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10005 (1415768037299060736)]} 0 77
[junit4:junit4]   2> 13005 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=35 
[junit4:junit4]   2> 13016 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10005 (1415768037385043968)]} 0 58
[junit4:junit4]   2> 13073 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=36 
[junit4:junit4]   2> 13074 T47 C3 P22421 /update {wt=javabin&version=2} {add=[6 (1415768037333663744)]} 0 164
[junit4:junit4]   2> 13076 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10006 (1415768037504581632)]} 0 4
[junit4:junit4]   2> 13080 T32 C4 P22410 /update {wt=javabin&version=2} {add=[7 (1415768037509824512)]} 0 2
[junit4:junit4]   2> 13090 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13091 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10006 (1415768037512970240)]} 0 10
[junit4:junit4]   2> 13093 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 13094 T46 C3 P22421 /update {wt=javabin&version=2} {add=[7 (1415768037516115968)]} 0 11
[junit4:junit4]   2> 13098 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[4 (-1415768037530796032)]} 0 1
[junit4:junit4]   2> 13100 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10007 (1415768037528698880)]} 0 5
[junit4:junit4]   2> 13106 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13107 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[4 (-1415768037533941760)]} 0 6
[junit4:junit4]   2> 13116 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 13116 T32 C4 P22410 /update {wt=javabin&version=2} {add=[8 (1415768037545476096)]} 0 5
[junit4:junit4]   2> 13117 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10007 (1415768037538136064)]} 0 12
[junit4:junit4]   2> 13121 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10004 (-1415768037554913280)]} 0 0
[junit4:junit4]   2> 13129 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 13130 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10004 (-1415768037560156160)]} 0 5
[junit4:junit4]   2> 13131 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 13132 T47 C3 P22421 /update {wt=javabin&version=2} {add=[8 (1415768037554913280)]} 0 11
[junit4:junit4]   2> 13136 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[5 (-1415768037570641920)]} 0 1
[junit4:junit4]   2> 13139 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10008 (1415768037568544768)]} 0 5
[junit4:junit4]   2> 13144 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13146 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[5 (-1415768037574836224)]} 0 7
[junit4:junit4]   2> 13153 T32 C4 P22410 /update {wt=javabin&version=2} {add=[9 (1415768037585321984)]} 0 3
[junit4:junit4]   2> 13156 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 13158 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10008 (1415768037579030528)]} 0 14
[junit4:junit4]   2> 13164 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10005 (-1415768037598953472)]} 0 1
[junit4:junit4]   2> 13170 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 13171 T47 C3 P22421 /update {wt=javabin&version=2} {add=[9 (1415768037593710592)]} 0 14
[junit4:junit4]   2> 13172 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 13174 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10005 (-1415768037604196352)]} 0 6
[junit4:junit4]   2> 13176 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[7 (-1415768037612584960)]} 0 0
[junit4:junit4]   2> 13182 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10009 (1415768037614682112)]} 0 4
[junit4:junit4]   2> 13186 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 13187 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[7 (-1415768037617827840)]} 0 6
[junit4:junit4]   2> 13195 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10 (1415768037628313600)]} 0 4
[junit4:junit4]   2> 13198 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 13199 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10009 (1415768037623070720)]} 0 13
[junit4:junit4]   2> 13203 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10007 (-1415768037640896512)]} 0 0
[junit4:junit4]   2> 13209 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13210 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10 (1415768037637750784)]} 0 10
[junit4:junit4]   2> 13212 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 13213 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10007 (-1415768037645090816)]} 0 6
[junit4:junit4]   2> 13222 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10010 (1415768037655576576)]} 0 5
[junit4:junit4]   2> 13243 T29 C4 P22410 /update {wt=javabin&version=2} {add=[11 (1415768037653479424)]} 0 29
[junit4:junit4]   2> 13272 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 13274 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10010 (1415768037666062336)]} 0 47
[junit4:junit4]   2> 13280 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10011 (1415768037719539712)]} 0 2
[junit4:junit4]   2> 13294 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13295 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10011 (1415768037726879744)]} 0 10
[junit4:junit4]   2> 13300 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10008 (-1415768037742608384)]} 0 0
[junit4:junit4]   2> 13301 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=44 
[junit4:junit4]   2> 13302 T46 C3 P22421 /update {wt=javabin&version=2} {add=[11 (1415768037688082432)]} 0 54
[junit4:junit4]   2> 13306 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[8 (-1415768037748899840)]} 0 0
[junit4:junit4]   2> 13310 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13312 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10008 (-1415768037747851264)]} 0 7
[junit4:junit4]   2> 13315 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 13317 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[8 (-1415768037754142720)]} 0 7
[junit4:junit4]   2> 13319 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10012 (1415768037759385600)]} 0 4
[junit4:junit4]   2> 13324 T32 C4 P22410 /update {wt=javabin&version=2} {add=[12 (1415768037764628480)]} 0 3
[junit4:junit4]   2> 13333 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13334 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10012 (1415768037767774208)]} 0 11
[junit4:junit4]   2> 13338 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 13339 T47 C3 P22421 /update {wt=javabin&version=2} {add=[12 (1415768037771968512)]} 0 12
[junit4:junit4]   2> 13340 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10013 (1415768037782454272)]} 0 3
[junit4:junit4]   2> 13347 T32 C4 P22410 /update {wt=javabin&version=2} {add=[13 (1415768037788745728)]} 0 3
[junit4:junit4]   2> 13356 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 13358 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10013 (1415768037788745728)]} 0 14
[junit4:junit4]   2> 13359 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13360 T46 C3 P22421 /update {wt=javabin&version=2} {add=[13 (1415768037796085760)]} 0 9
[junit4:junit4]   2> 13366 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10014 (1415768037808668672)]} 0 3
[junit4:junit4]   2> 13367 T29 C4 P22410 /update {wt=javabin&version=2} {add=[14 (1415768037810765824)]} 0 2
[junit4:junit4]   2> 13379 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13380 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10014 (1415768037817057280)]} 0 9
[junit4:junit4]   2> 13384 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10009 (-1415768037830688768)]} 0 0
[junit4:junit4]   2> 13385 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 13386 T47 C3 P22421 /update {wt=javabin&version=2} {add=[14 (1415768037817057281)]} 0 15
[junit4:junit4]   2> 13390 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[9 (-1415768037836980224)]} 0 1
[junit4:junit4]   2> 13392 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13393 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10009 (-1415768037834883072)]} 0 6
[junit4:junit4]   2> 13398 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13399 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[9 (-1415768037841174528)]} 0 6
[junit4:junit4]   2> 13400 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10015 (1415768037845368832)]} 0 3
[junit4:junit4]   2> 13416 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 13418 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10015 (1415768037851660288)]} 0 14
[junit4:junit4]   2> 13422 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10011 (-1415768037870534656)]} 0 0
[junit4:junit4]   2> 13425 T32 C4 P22410 /update {wt=javabin&version=2} {add=[15 (1415768037851660288)]} 0 21
[junit4:junit4]   2> 13431 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13433 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10011 (-1415768037874728960)]} 0 7
[junit4:junit4]   2> 13442 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10016 (1415768037886263296)]} 0 5
[junit4:junit4]   2> 13464 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=8 
[junit4:junit4]   2> 13466 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10016 (1415768037896749056)]} 0 19
[junit4:junit4]   2> 13470 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10012 (-1415768037920866304)]} 0 1
[junit4:junit4]   2> 13477 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 13478 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10012 (-1415768037925060608)]} 0 4
[junit4:junit4]   2> 13486 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10017 (1415768037934497792)]} 0 3
[junit4:junit4]   2> 13526 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 13527 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10017 (1415768037941837824)]} 0 38
[junit4:junit4]   2> 13529 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=75 
[junit4:junit4]   2> 13531 T46 C3 P22421 /update {wt=javabin&version=2} {add=[15 (1415768037877874688)]} 0 103
[junit4:junit4]   2> 13533 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10018 (1415768037984829440)]} 0 2
[junit4:junit4]   2> 13535 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[11 (-1415768037989023744)]} 0 1
[junit4:junit4]   2> 13544 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13545 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[11 (-1415768037993218048)]} 0 7
[junit4:junit4]   2> 13547 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13549 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10018 (1415768037991120896)]} 0 11
[junit4:junit4]   2> 13551 T29 C4 P22410 /update {wt=javabin&version=2} {add=[16 (1415768038003703808)]} 0 3
[junit4:junit4]   2> 13555 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10019 (1415768038007898112)]} 0 3
[junit4:junit4]   2> 13567 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 13569 T46 C3 P22421 /update {wt=javabin&version=2} {add=[16 (1415768038011043840)]} 0 13
[junit4:junit4]   2> 13571 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 13573 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10019 (1415768038014189568)]} 0 14
[junit4:junit4]   2> 13574 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[12 (-1415768038029918208)]} 0 0
[junit4:junit4]   2> 13577 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10014 (-1415768038033063936)]} 0 1
[junit4:junit4]   2> 13583 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13584 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[12 (-1415768038034112512)]} 0 6
[junit4:junit4]   2> 13584 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 13586 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10014 (-1415768038037258240)]} 0 6
[junit4:junit4]   2> 13591 T29 C4 P22410 /update {wt=javabin&version=2} {add=[17 (1415768038045646848)]} 0 3
[junit4:junit4]   2> 13594 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10020 (1415768038046695424)]} 0 4
[junit4:junit4]   2> 13608 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13610 T46 C3 P22421 /update {wt=javabin&version=2} {add=[17 (1415768038054035456)]} 0 14
[junit4:junit4]   2> 13612 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 13613 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10020 (1415768038056132608)]} 0 14
[junit4:junit4]   2> 13617 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10016 (-1415768038075006976)]} 0 0
[junit4:junit4]   2> 13619 T32 C4 P22410 /update {wt=javabin&version=2} {add=[18 (1415768038072909824)]} 0 5
[junit4:junit4]   2> 13627 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13629 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10016 (-1415768038080249856)]} 0 7
[junit4:junit4]   2> 13636 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 13637 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10021 (1415768038092832768)]} 0 4
[junit4:junit4]   2> 13638 T46 C3 P22421 /update {wt=javabin&version=2} {add=[18 (1415768038083395584)]} 0 14
[junit4:junit4]   2> 13650 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13651 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10021 (1415768038102269952)]} 0 9
[junit4:junit4]   2> 13660 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10022 (1415768038115901440)]} 0 5
[junit4:junit4]   2> 13678 T32 C4 P22410 /update {wt=javabin&version=2} {add=[19 (1415768038102269952)]} 0 35
[junit4:junit4]   2> 13708 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 13709 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10022 (1415768038125338624)]} 0 45
[junit4:junit4]   2> 13713 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10017 (-1415768038175670272)]} 0 1
[junit4:junit4]   2> 13715 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=26 
[junit4:junit4]   2> 13716 T46 C3 P22421 /update {wt=javabin&version=2} {add=[19 (1415768038144212992)]} 0 33
[junit4:junit4]   2> 13720 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[14 (-1415768038183010304)]} 0 1
[junit4:junit4]   2> 13720 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13722 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10017 (-1415768038179864576)]} 0 6
[junit4:junit4]   2> 13727 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13728 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[14 (-1415768038187204608)]} 0 5
[junit4:junit4]   2> 13729 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10023 (1415768038189301760)]} 0 4
[junit4:junit4]   2> 13734 T29 C4 P22410 /update {wt=javabin&version=2} {add=[20 (1415768038195593216)]} 0 2
[junit4:junit4]   2> 13740 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13742 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10023 (1415768038196641792)]} 0 10
[junit4:junit4]   2> 13746 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13746 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10018 (-1415768038210273280)]} 0 1
[junit4:junit4]   2> 13747 T47 C3 P22421 /update {wt=javabin&version=2} {add=[20 (1415768038201884672)]} 0 10
[junit4:junit4]   2> 13751 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[16 (-1415768038214467584)]} 0 1
[junit4:junit4]   2> 13753 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13755 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10018 (-1415768038214467584)]} 0 6
[junit4:junit4]   2> 13759 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13760 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[16 (-1415768038219710464)]} 0 5
[junit4:junit4]   2> 13760 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10024 (1415768038223904768)]} 0 2
[junit4:junit4]   2> 13767 T29 C4 P22410 /update {wt=javabin&version=2} {add=[21 (1415768038230196224)]} 0 3
[junit4:junit4]   2> 13773 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13774 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10024 (1415768038230196224)]} 0 10
[junit4:junit4]   2> 13779 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13780 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10025 (1415768038243827712)]} 0 2
[junit4:junit4]   2> 13780 T46 C3 P22421 /update {wt=javabin&version=2} {add=[21 (1415768038236487680)]} 0 10
[junit4:junit4]   2> 13786 T29 C4 P22410 /update {wt=javabin&version=2} {add=[22 (1415768038250119168)]} 0 3
[junit4:junit4]   2> 13792 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13793 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10025 (1415768038250119168)]} 0 9
[junit4:junit4]   2> 13797 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10020 (-1415768038263750656)]} 0 1
[junit4:junit4]   2> 13798 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13799 T47 C3 P22421 /update {wt=javabin&version=2} {add=[22 (1415768038256410624)]} 0 10
[junit4:junit4]   2> 13803 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[17 (-1415768038270042112)]} 0 1
[junit4:junit4]   2> 13805 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13806 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10020 (-1415768038268993536)]} 0 5
[junit4:junit4]   2> 13810 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13811 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[17 (-1415768038274236416)]} 0 5
[junit4:junit4]   2> 13812 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10026 (1415768038277382144)]} 0 3
[junit4:junit4]   2> 13818 T29 C4 P22410 /update {wt=javabin&version=2} {add=[23 (1415768038283673600)]} 0 3
[junit4:junit4]   2> 13824 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13825 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10026 (1415768038283673600)]} 0 10
[junit4:junit4]   2> 13829 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13831 T46 C3 P22421 /update {wt=javabin&version=2} {add=[23 (1415768038289965056)]} 0 9
[junit4:junit4]   2> 13835 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[18 (-1415768038303596544)]} 0 0
[junit4:junit4]   2> 13843 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13844 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[18 (-1415768038308839424)]} 0 4
[junit4:junit4]   2> 13853 T32 C4 P22410 /update {wt=javabin&version=2} {add=[24 (1415768038317228032)]} 0 6
[junit4:junit4]   2> 13854 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10027 (1415768038298353664)]} 0 25
[junit4:junit4]   2> 13935 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 13936 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=68 
[junit4:junit4]   2> 13936 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10027 (1415768038327713792)]} 0 79
[junit4:junit4]   2> 13937 T46 C3 P22421 /update {wt=javabin&version=2} {add=[24 (1415768038327713793)]} 0 79
[junit4:junit4]   2> 13942 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10028 (1415768038413697024)]} 0 3
[junit4:junit4]   2> 13943 T32 C4 P22410 /update {wt=javabin&version=2} {add=[25 (1415768038414745600)]} 0 3
[junit4:junit4]   2> 13953 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13954 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13954 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10028 (1415768038419988480)]} 0 9
[junit4:junit4]   2> 13955 T47 C3 P22421 /update {wt=javabin&version=2} {add=[25 (1415768038421037056)]} 0 8
[junit4:junit4]   2> 13958 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10021 (-1415768038432571392)]} 0 1
[junit4:junit4]   2> 13959 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[20 (-1415768038432571393)]} 0 1
[junit4:junit4]   2> 13965 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13966 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13966 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10021 (-1415768038435717120)]} 0 5
[junit4:junit4]   2> 13968 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[20 (-1415768038437814272)]} 0 6
[junit4:junit4]   2> 13972 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10029 (1415768038446202880)]} 0 2
[junit4:junit4]   2> 13974 T32 C4 P22410 /update {wt=javabin&version=2} {add=[26 (1415768038447251456)]} 0 3
[junit4:junit4]   2> 13983 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13984 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 13984 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10029 (1415768038451445760)]} 0 9
[junit4:junit4]   2> 13985 T47 C3 P22421 /update {wt=javabin&version=2} {add=[26 (1415768038453542912)]} 0 8
[junit4:junit4]   2> 13989 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10030 (1415768038464028672)]} 0 2
[junit4:junit4]   2> 13990 T29 C4 P22410 /update {wt=javabin&version=2} {add=[27 (1415768038465077248)]} 0 2
[junit4:junit4]   2> 13998 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13999 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10030 (1415768038468222976)]} 0 7
[junit4:junit4]   2> 14000 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 14001 T46 C3 P22421 /update {wt=javabin&version=2} {add=[27 (1415768038470320128)]} 0 8
[junit4:junit4]   2> 14003 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10022 (-1415768038479757312)]} 0 1
[junit4:junit4]   2> 14006 T32 C4 P22410 /update {wt=javabin&version=2} {add=[28 (1415768038481854464)]} 0 2
[junit4:junit4]   2> 14010 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14010 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10022 (-1415768038482903040)]} 0 4
[junit4:junit4]   2> 14015 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10031 (1415768038490243072)]} 0 2
[junit4:junit4]   2> 14016 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14016 T47 C3 P22421 /update {wt=javabin&version=2} {add=[28 (1415768038487097344)]} 0 7
[junit4:junit4]   2> 14020 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[21 (-1415768038497583104)]} 0 1
[junit4:junit4]   2> 14024 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14025 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10031 (1415768038496534528)]} 0 7
[junit4:junit4]   2> 14027 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14027 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[21 (-1415768038500728832)]} 0 4
[junit4:junit4]   2> 14029 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10023 (-1415768038505971712)]} 0 1
[junit4:junit4]   2> 14036 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14036 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10023 (-1415768038510166016)]} 0 4
[junit4:junit4]   2> 14045 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10032 (1415768038519603200)]} 0 5
[junit4:junit4]   2> 14057 T29 C4 P22410 /update {wt=javabin&version=2} {add=[29 (1415768038509117440)]} 0 26
[junit4:junit4]   2> 14084 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 14086 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10032 (1415768038538477568)]} 0 28
[junit4:junit4]   2> 14087 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=21 
[junit4:junit4]   2> 14088 T46 C3 P22421 /update {wt=javabin&version=2} {add=[29 (1415768038540574720)]} 0 28
[junit4:junit4]   2> 14091 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10033 (1415768038570983424)]} 0 2
[junit4:junit4]   2> 14093 T32 C4 P22410 /update {wt=javabin&version=2} {add=[30 (1415768038573080576)]} 0 2
[junit4:junit4]   2> 14102 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14103 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10033 (1415768038577274880)]} 0 8
[junit4:junit4]   2> 14104 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14105 T47 C3 P22421 /update {wt=javabin&version=2} {add=[30 (1415768038578323456)]} 0 9
[junit4:junit4]   2> 14109 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[22 (-1415768038589857793)]} 0 1
[junit4:junit4]   2> 14109 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10034 (1415768038589857792)]} 0 2
[junit4:junit4]   2> 14116 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14117 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[22 (-1415768038595100672)]} 0 5
[junit4:junit4]   2> 14119 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14120 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10034 (1415768038595100673)]} 0 8
[junit4:junit4]   2> 14122 T32 C4 P22410 /update {wt=javabin&version=2} {add=[31 (1415768038603489280)]} 0 2
[junit4:junit4]   2> 14125 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10035 (1415768038606635008)]} 0 2
[junit4:junit4]   2> 14137 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=8 
[junit4:junit4]   2> 14138 T46 C3 P22421 /update {wt=javabin&version=2} {add=[31 (1415768038607683584)]} 0 13
[junit4:junit4]   2> 14140 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14141 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10035 (1415768038617120768)]} 0 13
[junit4:junit4]   2> 14142 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[23 (-1415768038625509376)]} 0 1
[junit4:junit4]   2> 14145 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10025 (-1415768038628655104)]} 0 1
[junit4:junit4]   2> 14149 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14150 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[23 (-1415768038628655104)]} 0 5
[junit4:junit4]   2> 14152 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14152 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10025 (-1415768038631800832)]} 0 4
[junit4:junit4]   2> 14155 T32 C4 P22410 /update {wt=javabin&version=2} {add=[32 (1415768038638092288)]} 0 2
[junit4:junit4]   2> 14157 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10036 (1415768038640189440)]} 0 2
[junit4:junit4]   2> 14165 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 14166 T46 C3 P22421 /update {wt=javabin&version=2} {add=[32 (1415768038642286592)]} 0 8
[junit4:junit4]   2> 14167 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14168 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10036 (1415768038645432320)]} 0 8
[junit4:junit4]   2> 14172 T29 C4 P22410 /update {wt=javabin&version=2} {add=[33 (1415768038654869504)]} 0 3
[junit4:junit4]   2> 14182 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14183 T47 C3 P22421 /update {wt=javabin&version=2} {add=[33 (1415768038661160960)]} 0 8
[junit4:junit4]   2> 14191 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10037 (1415768038655918080)]} 0 20
[junit4:junit4]   2> 14192 T29 C4 P22410 /update {wt=javabin&version=2} {add=[34 (1415768038673743872)]} 0 5
[junit4:junit4]   2> 14223 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 14224 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10037 (1415768038682132480)]} 0 29
[junit4:junit4]   2> 14226 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=21 
[junit4:junit4]   2> 14227 T46 C3 P22421 /update {wt=javabin&version=2} {add=[34 (1415768038682132481)]} 0 32
[junit4:junit4]   2> 14228 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10028 (-1415768038714638336)]} 0 1
[junit4:junit4]   2> 14232 T32 C4 P22410 /update {wt=javabin&version=2} {add=[35 (1415768038718832640)]} 0 2
[junit4:junit4]   2> 14234 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14235 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10028 (-1415768038717784064)]} 0 4
[junit4:junit4]   2> 14239 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10038 (1415768038726172672)]} 0 2
[junit4:junit4]   2> 14247 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=8 
[junit4:junit4]   2> 14248 T47 C3 P22421 /update {wt=javabin&version=2} {add=[35 (1415768038724075520)]} 0 13
[junit4:junit4]   2> 14252 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[25 (-1415768038740852736)]} 0 1
[junit4:junit4]   2> 14259 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14260 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[25 (-1415768038745047040)]} 0 5
[junit4:junit4]   2> 14265 T32 C4 P22410 /update {wt=javabin&version=2} {add=[36 (1415768038753435648)]} 0 2
[junit4:junit4]   2> 14271 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14288 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10038 (1415768038736658432)]} 0 41
[junit4:junit4]   2> 14291 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 14292 T47 C3 P22421 /update {wt=javabin&version=2} {add=[36 (1415768038759727104)]} 0 23
[junit4:junit4]   2> 14293 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10039 (1415768038782795776)]} 0 2
[junit4:junit4]   2> 14298 T29 C4 P22410 /update {wt=javabin&version=2} {add=[37 (1415768038786990080)]} 0 3
[junit4:junit4]   2> 14303 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 14304 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10039 (1415768038788038656)]} 0 8
[junit4:junit4]   2> 14308 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14309 T46 C3 P22421 /update {wt=javabin&version=2} {add=[37 (1415768038792232960)]} 0 7
[junit4:junit4]   2> 14309 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10040 (1415768038798524416)]} 0 2
[junit4:junit4]   2> 14313 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[28 (-1415768038803767296)]} 0 1
[junit4:junit4]   2> 14319 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 14320 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10040 (1415768038804815872)]} 0 8
[junit4:junit4]   2> 14322 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14323 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[28 (-1415768038810058752)]} 0 5
[junit4:junit4]   2> 14325 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10041 (1415768038816350208)]} 0 2
[junit4:junit4]   2> 14328 T29 C4 P22410 /update {wt=javabin&version=2} {add=[38 (1415768038819495936)]} 0 2
[junit4:junit4]   2> 14334 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14336 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10041 (1415768038821593088)]} 0 8
[junit4:junit4]   2> 14339 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14340 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10030 (-1415768038832078848)]} 0 1
[junit4:junit4]   2> 14340 T46 C3 P22421 /update {wt=javabin&version=2} {add=[38 (1415768038825787392)]} 0 8
[junit4:junit4]   2> 14346 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14347 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10030 (-1415768038835224576)]} 0 5
[junit4:junit4]   2> 14355 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10042 (1415768038845710336)]} 0 4
[junit4:junit4]   2> 14551 T29 C4 P22410 /update {wt=javabin&version=2} {add=[39 (1415768038836273152)]} 0 208
[junit4:junit4]   2> 14563 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 14564 T46 C3 P22421 /update {wt=javabin&version=2} {add=[39 (1415768039057522688)]} 0 10
[junit4:junit4]   2> 14570 T29 C4 P22410 /update {wt=javabin&version=2} {add=[40 (1415768039072202752)]} 0 3
[junit4:junit4]   2> 14580 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14581 T46 C3 P22421 /update {wt=javabin&version=2} {add=[40 (1415768039079542784)]} 0 7
[junit4:junit4]   2> 14587 T29 C4 P22410 /update {wt=javabin&version=2} {add=[41 (1415768039090028544)]} 0 2
[junit4:junit4]   2> 14598 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 14599 T46 C3 P22421 /update {wt=javabin&version=2} {add=[41 (1415768039097368576)]} 0 8
[junit4:junit4]   2> 14603 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[30 (-1415768039108902912)]} 0 0
[junit4:junit4]   2> 14612 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14614 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[30 (-1415768039114145792)]} 0 6
[junit4:junit4]   2> 14620 T29 C4 P22410 /update {wt=javabin&version=2} {add=[42 (1415768039124631552)]} 0 2
[junit4:junit4]   2> 14631 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14632 T46 C3 P22421 /update {wt=javabin&version=2} {add=[42 (1415768039131971584)]} 0 8
[junit4:junit4]   2> 14638 T66 C2 P22439 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 14638 T29 C4 P22410 /update {wt=javabin&version=2} {add=[43 (1415768039143505920)]} 0 3
[junit4:junit4]   2> 14755 T66 C2 P22439 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty2/tlog/tlog.0000000000000000000 refcount=2}}
[junit4:junit4]   2> 14755 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=203 
[junit4:junit4]   2> 14756 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=110 
[junit4:junit4]   2> 14756 T66 C2 P22439 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:22421/solr/collection1/. core=collection1
[junit4:junit4]   2> 14757 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10042 (1415768038853050368)]} 0 399
[junit4:junit4]   2> 14757 T66 C2 P22439 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14757 T46 C3 P22421 /update {wt=javabin&version=2} {add=[43 (1415768039149797376)]} 0 116
[junit4:junit4]   2> 14762 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10043 (1415768039274577920)]} 0 2
[junit4:junit4]   2> 14763 T32 C4 P22410 /update {wt=javabin&version=2} {add=[44 (1415768039275626496)]} 0 2
[junit4:junit4]   2> 14780 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14781 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14781 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10043 (1415768039289257985)]} 0 7
[junit4:junit4]   2> 14782 T47 C3 P22421 /update {wt=javabin&version=2} {add=[44 (1415768039289257984)]} 0 8
[junit4:junit4]   2> 14786 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[32 (-1415768039299743745)]} 0 1
[junit4:junit4]   2> 14786 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10044 (1415768039299743744)]} 0 1
[junit4:junit4]   2> 14793 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14794 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[32 (-1415768039304986624)]} 0 5
[junit4:junit4]   2> 14795 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14796 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10044 (1415768039304986625)]} 0 6
[junit4:junit4]   2> 14796 T48 C3 P22421 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14799 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10032 (-1415768039313375233)]} 0 1
[junit4:junit4]   2> 14799 T32 C4 P22410 /update {wt=javabin&version=2} {add=[45 (1415768039313375232)]} 0 1
[junit4:junit4]   2> 14807 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14808 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10032 (-1415768039318618112)]} 0 6
[junit4:junit4]   2> 14811 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14812 T47 C3 P22421 /update {wt=javabin&version=2} {add=[45 (1415768039318618113)]} 0 10
[junit4:junit4]   2> 14822 T29 C4 P22410 /update {wt=javabin&version=2} {add=[46 (1415768039332249600)]} 0 6
[junit4:junit4]   2> 14831 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14832 T47 C3 P22421 /update {wt=javabin&version=2} {add=[46 (1415768039342735360)]} 0 7
[junit4:junit4]   2> 14836 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[33 (-1415768039352172544)]} 0 1
[junit4:junit4]   2> 14843 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14844 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[33 (-1415768039356366848)]} 0 5
[junit4:junit4]   2> 14849 T29 C4 P22410 /update {wt=javabin&version=2} {add=[47 (1415768039365804032)]} 0 2
[junit4:junit4]   2> 14858 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14860 T47 C3 P22421 /update {wt=javabin&version=2} {add=[47 (1415768039371046912)]} 0 7
[junit4:junit4]   2> 14864 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[35 (-1415768039381532672)]} 0 1
[junit4:junit4]   2> 14871 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14872 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[35 (-1415768039385726976)]} 0 5
[junit4:junit4]   2> 14877 T29 C4 P22410 /update {wt=javabin&version=2} {add=[48 (1415768039395164160)]} 0 2
[junit4:junit4]   2> 14885 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14886 T47 C3 P22421 /update {wt=javabin&version=2} {add=[48 (1415768039400407040)]} 0 6
[junit4:junit4]   2> 14891 T29 C4 P22410 /update {wt=javabin&version=2} {add=[49 (1415768039409844224)]} 0 1
[junit4:junit4]   2> 14899 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14900 T47 C3 P22421 /update {wt=javabin&version=2} {add=[49 (1415768039415087104)]} 0 6
[junit4:junit4]   2> 14905 T29 C4 P22410 /update {wt=javabin&version=2} {add=[50 (1415768039424524288)]} 0 2
[junit4:junit4]   2> 14915 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14916 T47 C3 P22421 /update {wt=javabin&version=2} {add=[50 (1415768039430815744)]} 0 7
[junit4:junit4]   2> 14920 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[38 (-1415768039440252928)]} 0 1
[junit4:junit4]   2> 14925 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10045 (1415768039328055296)]} 0 114
[junit4:junit4]   2> 14926 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14927 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[38 (-1415768039444447232)]} 0 4
[junit4:junit4]   2> 14932 T32 C4 P22410 /update {wt=javabin&version=2} {add=[51 (1415768039451787264)]} 0 2
[junit4:junit4]   2> 14935 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14936 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10045 (1415768039451787264)]} 0 6
[junit4:junit4]   2> 14940 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14940 T47 C3 P22421 /update {wt=javabin&version=2} {add=[51 (1415768039458078720)]} 0 5
[junit4:junit4]   2> 14941 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10046 (1415768039462273024)]} 0 2
[junit4:junit4]   2> 14945 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[39 (-1415768039467515904)]} 0 1
[junit4:junit4]   2> 14949 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14950 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10046 (1415768039467515904)]} 0 6
[junit4:junit4]   2> 14952 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14953 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[39 (-1415768039471710208)]} 0 5
[junit4:junit4]   2> 14954 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10033 (-1415768039476953088)]} 0 1
[junit4:junit4]   2> 14958 T32 C4 P22410 /update {wt=javabin&version=2} {add=[52 (1415768039480098816)]} 0 2
[junit4:junit4]   2> 14961 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14962 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10033 (-1415768039481147392)]} 0 4
[junit4:junit4]   2> 14967 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10047 (1415768039489536000)]} 0 2
[junit4:junit4]   2> 14977 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14978 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10047 (1415768039494778880)]} 0 8
[junit4:junit4]   2> 14982 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10035 (-1415768039505264640)]} 0 1
[junit4:junit4]   2> 14987 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14988 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10035 (-1415768039507361792)]} 0 5
[junit4:junit4]   2> 15008 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15009 T47 C3 P22421 /update {wt=javabin&version=2} {add=[52 (1415768039485341696)]} 0 48
[junit4:junit4]   2> 15018 T29 C4 P22410 /update {wt=javabin&version=2} {add=[53 (1415768039538819072)]} 0 6
[junit4:junit4]   2> 15026 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15027 T47 C3 P22421 /update {wt=javabin&version=2} {add=[53 (1415768039548256256)]} 0 6
[junit4:junit4]   2> 15031 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10048 (1415768039516798976)]} 0 40
[junit4:junit4]   2> 15033 T29 C4 P22410 /update {wt=javabin&version=2} {add=[54 (1415768039557693440)]} 0 2
[junit4:junit4]   2> 15039 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15040 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10048 (1415768039561887744)]} 0 6
[junit4:junit4]   2> 15041 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15042 T46 C3 P22421 /update {wt=javabin&version=2} {add=[54 (1415768039562936320)]} 0 6
[junit4:junit4]   2> 15045 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10049 (1415768039571324928)]} 0 2
[junit4:junit4]   2> 15046 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[40 (-1415768039573422080)]} 0 1
[junit4:junit4]   2> 15053 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15054 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15054 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[40 (-1415768039576567809)]} 0 5
[junit4:junit4]   2> 15054 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10049 (1415768039576567808)]} 0 6
[junit4:junit4]   2> 15059 T32 C4 P22410 /update {wt=javabin&version=2} {add=[55 (1415768039586004992)]} 0 2
[junit4:junit4]   2> 15060 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10050 (1415768039586004993)]} 0 3
[junit4:junit4]   2> 15067 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15068 T46 C3 P22421 /update {wt=javabin&version=2} {add=[55 (1415768039591247872)]} 0 6
[junit4:junit4]   2> 15068 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15069 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10050 (1415768039592296448)]} 0 6
[junit4:junit4]   2> 15072 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[41 (-1415768039599636480)]} 0 1
[junit4:junit4]   2> 15072 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10038 (-1415768039600685056)]} 0 0
[junit4:junit4]   2> 15079 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15079 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15080 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10038 (-1415768039604879361)]} 0 5
[junit4:junit4]   2> 15080 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[41 (-1415768039604879360)]} 0 5
[junit4:junit4]   2> 15085 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10051 (1415768039613267968)]} 0 2
[junit4:junit4]   2> 15086 T29 C4 P22410 /update {wt=javabin&version=2} {add=[56 (1415768039613267969)]} 0 3
[junit4:junit4]   2> 15091 T48 C3 P22421 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty1/index,segFN=segments_2,generation=2,filenames=[_9_MockRandom_0.tib, _8_nrm.cfe, _9_MockFixedIntBlock_0.skp, _8_MockFixedIntBlock_0.tii, _8_MockRandom_0.tip, _8_Asserting_0.frq, _8_MockRandom_0.sd, _8_MockFixedIntBlock_0.pyl, _8_MockFixedIntBlock_0.tib, _9_MockRandom_0.tii, _8_1.del, _8.si, _9_MockFixedIntBlock_0.doc, _9_MockFixedIntBlock_0.frq, _9_nrm.cfs, _9.si, _9_MockFixedIntBlock_0.pyl, _8_MockRandom_0.tim, _9_nrm.cfe, _8_Asserting_0.tim, _9.fdt, _9_MockRandom_0.sd, _8_Asserting_0.tip, _9.fdx, _9_SimpleText_0.pst, _9_MockFixedIntBlock_0.pos, _8_nrm.cfs, _8.fnm, _9_MockFixedIntBlock_0.tii, _8_MockRandom_0.prx, _8_MockFixedIntBlock_0.doc, _9_MockFixedIntBlock_0.tib, _8_MockFixedIntBlock_0.skp, _9_Asserting_0.tip, _9_MockRandom_0.frq, _9_Asserting_0.tim, _8_MockFixedIntBlock_0.frq, _8_MockRandom_0.frq, _9_Asserting_0.frq, _9.fnm, _8_Asserting_0.prx, _9_MockRandom_0.doc, _9_Asserting_0.prx, _9_MockRandom_0.pyl, _9_MockRandom_0.pos, segments_2, _8.fdx, _8_MockFixedIntBlock_0.pos, _8_SimpleText_0.pst, _9_MockRandom_0.skp, _8.fdt]
[junit4:junit4]   2> 15092 T48 C3 P22421 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 15093 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15094 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10051 (1415768039618510848)]} 0 6
[junit4:junit4]   2> 15094 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15095 T46 C3 P22421 /update {wt=javabin&version=2} {add=[56 (1415768039618510849)]} 0 7
[junit4:junit4]   2> 15098 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10039 (-1415768039626899456)]} 0 1
[junit4:junit4]   2> 15098 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[42 (-1415768039627948032)]} 0 0
[junit4:junit4]   2> 15103 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15104 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15104 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[42 (-1415768039631093760)]} 0 3
[junit4:junit4]   2> 15104 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10039 (-1415768039630045184)]} 0 4
[junit4:junit4]   2> 15109 T32 C4 P22410 /update {wt=javabin&version=2} {add=[57 (1415768039637385216)]} 0 2
[junit4:junit4]   2> 15109 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10052 (1415768039638433792)]} 0 1
[junit4:junit4]   2> 15118 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15119 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15122 T46 C3 P22421 /update {wt=javabin&version=2} {add=[57 (1415768039642628096)]} 0 10
[junit4:junit4]   2> 15127 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[43 (-1415768039658356736)]} 0 0
[junit4:junit4]   2> 15130 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10052 (1415768039642628097)]} 0 18
[junit4:junit4]   2> 15135 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15136 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[43 (-1415768039662551040)]} 0 5
[junit4:junit4]   2> 15144 T32 C4 P22410 /update {wt=javabin&version=2} {add=[58 (1415768039671988224)]} 0 5
[junit4:junit4]   2> 15151 T48 C3 P22421 oass.SolrIndexSearcher.<init> Opening Searcher@30322765 realtime
[junit4:junit4]   2> 15152 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10053 (1415768039664648192)]} 0 19
[junit4:junit4]   2> 15153 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15153 T48 C3 P22421 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15154 T46 C3 P22421 /update {wt=javabin&version=2} {add=[58 (1415768039680376832)]} 0 7
[junit4:junit4]   2> 15154 T48 C3 P22421 /update {waitSearcher=true&openSearcher=false&commit=true&commit_end_point=true&wt=javabin&softCommit=false&version=2} {commit=} 0 358
[junit4:junit4]   2> 15158 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[45 (-1415768039690862592)]} 0 1
[junit4:junit4]   2> 15161 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15163 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10053 (1415768039688765440)]} 0 7
[junit4:junit4]   2> 15166 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15167 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[45 (-1415768039696105472)]} 0 5
[junit4:junit4]   2> 15168 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10054 (1415768039700299776)]} 0 2
[junit4:junit4]   2> 15174 T32 C4 P22410 /update {wt=javabin&version=2} {add=[59 (1415768039705542656)]} 0 2
[junit4:junit4]   2> 15177 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15178 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10054 (1415768039705542656)]} 0 7
[junit4:junit4]   2> 15182 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10040 (-1415768039716028416)]} 0 0
[junit4:junit4]   2> 15185 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15186 T47 C3 P22421 /update {wt=javabin&version=2} {add=[59 (1415768039712882688)]} 0 7
[junit4:junit4]   2> 15189 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15191 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10040 (-1415768039719174144)]} 0 6
[junit4:junit4]   2> 15191 T32 C4 P22410 /update {wt=javabin&version=2} {add=[60 (1415768039723368448)]} 0 2
[junit4:junit4]   2> 15197 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10055 (1415768039729659904)]} 0 2
[junit4:junit4]   2> 15200 T66 C2 P22439 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15201 T66 C2 P22439 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 15201 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15203 T46 C3 P22421 /update {wt=javabin&version=2} {add=[60 (1415768039729659904)]} 0 8
[junit4:junit4]   2> 15207 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[47 (-1415768039742242816)]} 0 1
[junit4:junit4]   2> 15208 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15209 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10055 (1415768039736999936)]} 0 8
[junit4:junit4]   2> 15214 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10041 (-1415768039749582848)]} 0 1
[junit4:junit4]   2> 15214 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15215 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[47 (-1415768039746437120)]} 0 4
[junit4:junit4]   2> 15221 T66 C2 P22439 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15221 T29 C4 P22410 /update {wt=javabin&version=2} {add=[61 (1415768039755874304)]} 0 2
[junit4:junit4]   2> 15222 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15224 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10041 (-1415768039753777152)]} 0 6
[junit4:junit4]   2> 15229 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10056 (1415768039764262912)]} 0 2
[junit4:junit4]   2> 15230 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15232 T46 C3 P22421 /update {wt=javabin&version=2} {add=[61 (1415768039761117184)]} 0 8
[junit4:junit4]   2> 15236 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[48 (-1415768039772651520)]} 0 1
[junit4:junit4]   2> 15243 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15244 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[48 (-1415768039776845824)]} 0 4
[junit4:junit4]   2> 15251 T29 C4 P22410 /update {wt=javabin&version=2} {add=[62 (1415768039786283008)]} 0 2
[junit4:junit4]   2> 15263 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15264 T46 C3 P22421 /update {wt=javabin&version=2} {add=[62 (1415768039792574464)]} 0 9
[junit4:junit4]   2> 15269 T29 C4 P22410 /update {wt=javabin&version=2} {add=[63 (1415768039806205952)]} 0 2
[junit4:junit4]   2> 15273 T49 C3 P22421 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15273 T49 C3 P22421 REQ /replication {command=indexversion&wt=javabin&qt=/replication&version=2} status=0 QTime=50 
[junit4:junit4]   2> 15275 T66 C2 P22439 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15275 T66 C2 P22439 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15276 T66 C2 P22439 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15277 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15278 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10056 (1415768039770554368)]} 0 45
[junit4:junit4]   2> 15279 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15280 T46 C3 P22421 /update {wt=javabin&version=2} {add=[63 (1415768039812497408)]} 0 6
[junit4:junit4]   2> 15282 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[10042 (-1415768039819837440)]} 0 1
[junit4:junit4]   2> 15288 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15289 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10042 (-1415768039824031744)]} 0 5
[junit4:junit4]   2> 15294 T49 C3 P22421 REQ /replication {command=filelist&generation=2&wt=javabin&qt=/replication&version=2} status=0 QTime=16 
[junit4:junit4]   2> 15296 T66 C2 P22439 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 51
[junit4:junit4]   2> 15298 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10057 (1415768039832420352)]} 0 6
[junit4:junit4]   2> 15303 T66 C2 P22439 oash.SnapPuller.fetchLatestIndex Starting download to ./org.apache.solr.cloud.RecoveryZkTest-1350181602297/jetty2/index.20121014022657581 fullCopy=true
[junit4:junit4]   2> 15304 T32 C4 P22410 /update {wt=javabin&version=2} {add=[64 (1415768039821934592)]} 0 21
[junit4:junit4]   2> 15306 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15307 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10057 (1415768039841857536)]} 0 6
[junit4:junit4]   2> 15311 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10043 (-1415768039850246144)]} 0 1
[junit4:junit4]   2> 15312 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15313 T47 C3 P22421 /update {wt=javabin&version=2} {add=[64 (1415768039847100416)]} 0 6
[junit4:junit4]   2> 15315 T49 C3 P22421 REQ /replication {file=_9_MockRandom_0.tib&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=4 
[junit4:junit4]   2> 15318 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[54 (-1415768039857586176)]} 0 1
[junit4:junit4]   2> 15318 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15320 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10043 (-1415768039854440448)]} 0 6
[junit4:junit4]   2> 15324 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10058 (1415768039862829056)]} 0 2
[junit4:junit4]   2> 15324 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15326 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[54 (-1415768039861780480)]} 0 6
[junit4:junit4]   2> 15331 T32 C4 P22410 /update {wt=javabin&version=2} {add=[65 (1415768039870169088)]} 0 2
[junit4:junit4]   2> 15331 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15333 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10058 (1415768039867023360)]} 0 7
[junit4:junit4]   2> 15337 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10045 (-1415768039877509120)]} 0 1
[junit4:junit4]   2> 15337 T49 C3 P22421 REQ /replication {file=_9_MockFixedIntBlock_0.skp&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15339 T66 C2 P22439 oash.SnapPuller$FileFetcher.fetchPackets WARNING No content recieved for file: {name=_9_MockFixedIntBlock_0.skp, lastmodified=1350181617000, size=0}
[junit4:junit4]   2> 15341 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15342 T47 C3 P22421 /update {wt=javabin&version=2} {add=[65 (1415768039876460544)]} 0 7
[junit4:junit4]   2> 15342 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15343 T49 C3 P22421 REQ /replication {file=_8_nrm.cfe&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15344 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10045 (-1415768039880654848)]} 0 5
[junit4:junit4]   2> 15347 T32 C4 P22410 /update {wt=javabin&version=2} {add=[66 (1415768039887994880)]} 0 1
[junit4:junit4]   2> 15348 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10059 (1415768039889043456)]} 0 1
[junit4:junit4]   2> 15349 T49 C3 P22421 REQ /replication {file=_8_MockFixedIntBlock_0.tii&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15355 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15355 T49 C3 P22421 REQ /replication {file=_8_MockRandom_0.tip&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15356 T46 C3 P22421 /update {wt=javabin&version=2} {add=[66 (1415768039892189184)]} 0 6
[junit4:junit4]   2> 15358 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15359 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10059 (1415768039894286336)]} 0 8
[junit4:junit4]   2> 15360 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[55 (-1415768039901626368)]} 0 1
[junit4:junit4]   2> 15362 T49 C3 P22421 REQ /replication {file=_8_MockRandom_0.sd&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15365 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10060 (1415768039905820672)]} 0 3
[junit4:junit4]   2> 15366 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15367 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[55 (-1415768039905820672)]} 0 5
[junit4:junit4]   2> 15368 T49 C3 P22421 REQ /replication {file=_8_Asserting_0.frq&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15373 T32 C4 P22410 /update {wt=javabin&version=2} {add=[67 (1415768039915257856)]} 0 2
[junit4:junit4]   2> 15374 T49 C3 P22421 REQ /replication {file=_8_MockFixedIntBlock_0.pyl&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15375 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15376 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10060 (1415768039913160704)]} 0 6
[junit4:junit4]   2> 15376 T66 C2 P22439 oash.SnapPuller$FileFetcher.fetchPackets WARNING No content recieved for file: {name=_8_MockFixedIntBlock_0.pyl, lastmodified=1350181616000, size=0}
[junit4:junit4]   2> 15380 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10047 (-1415768039923646464)]} 0 1
[junit4:junit4]   2> 15381 T49 C3 P22421 REQ /replication {file=_8_MockFixedIntBlock_0.tib&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15388 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15389 T49 C3 P22421 REQ /replication {file=_9_MockRandom_0.tii&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15390 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[10047 (-1415768039928889344)]} 0 5
[junit4:junit4]   2> 15409 T49 C3 P22421 REQ /replication {file=_8_1.del&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15410 T32 C4 P22410 /update {wt=javabin&version=2} {add=[10061 (1415768039953006592)]} 0 2
[junit4:junit4]   2> 15416 T49 C3 P22421 REQ /replication {file=_9_MockFixedIntBlock_0.doc&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15423 T56 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15424 T49 C3 P22421 REQ /replication {file=_8.si&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15424 T46 C3 P22421 /update {wt=javabin&version=2} {add=[10061 (1415768039960346624)]} 0 9
[junit4:junit4]   2> 15424 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15433 T47 C3 P22421 /update {wt=javabin&version=2} {add=[67 (1415768039920500736)]} 0 56
[junit4:junit4]   2> 15435 T32 C4 P22410 /update {wt=javabin&version=2} {delete=[10048 (-1415768039980269568)]} 0 1
[junit4:junit4]   2> 15437 T49 C3 P22421 REQ /replication {file=_9_MockFixedIntBlock_0.frq&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15440 T29 C4 P22410 /update {wt=javabin&version=2} {add=[68 (1415768039983415296)]} 0 3
[junit4:junit4]   2> 15442 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15443 T47 C3 P22421 /update {wt=javabin&version=2} {delete=[10048 (-1415768039985512448)]} 0 5
[junit4:junit4]   2> 15444 T49 C3 P22421 REQ /replication {file=_9_nrm.cfs&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15450 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15451 T46 C3 P22421 /update {wt=javabin&version=2} {add=[68 (1415768039991803904)]} 0 7
[junit4:junit4]   2> 15452 T49 C3 P22421 REQ /replication {file=_9.si&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15459 T49 C3 P22421 REQ /replication {file=_9_MockFixedIntBlock_0.pyl&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15460 T32 C4 P22410 /update {wt=javabin&version=2} {add=[69 (1415768040002289664)]} 0 5
[junit4:junit4]   2> 15461 T66 C2 P22439 oash.SnapPuller$FileFetcher.fetchPackets WARNING No content recieved for file: {name=_9_MockFixedIntBlock_0.pyl, lastmodified=1350181617000, size=0}
[junit4:junit4]   2> 15467 T49 C3 P22421 REQ /replication {file=_8_MockRandom_0.tim&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15469 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15470 T46 C3 P22421 /update {wt=javabin&version=2} {add=[69 (1415768040010678272)]} 0 7
[junit4:junit4]   2> 15483 T49 C3 P22421 REQ /replication {file=_9_nrm.cfe&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15485 T32 C4 P22410 /update {wt=javabin&version=2} {add=[70 (1415768040032698368)]} 0 2
[junit4:junit4]   2> 15490 T49 C3 P22421 REQ /replication {file=_9.fdt&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15494 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10062 (1415768039994949632)]} 0 47
[junit4:junit4]   2> 15495 T49 C3 P22421 REQ /replication {file=_8_Asserting_0.tim&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15495 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15497 T46 C3 P22421 /update {wt=javabin&version=2} {add=[70 (1415768040038989824)]} 0 7
[junit4:junit4]   2> 15502 T49 C3 P22421 REQ /replication {file=_9_MockRandom_0.sd&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15502 T29 C4 P22410 /update {wt=javabin&version=2} {delete=[56 (-1415768040050524160)]} 0 1
[junit4:junit4]   2> 15536 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15538 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10062 (1415768040048427008)]} 0 40
[junit4:junit4]   2> 15540 T49 C3 P22421 REQ /replication {file=_8_Asserting_0.tip&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[junit4:junit4]   2> 15543 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15543 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10063 (1415768040093515776)]} 0 2
[junit4:junit4]   2> 15545 T46 C3 P22421 /update {wt=javabin&version=2} {delete=[56 (-1415768040090370048)]} 0 6
[junit4:junit4]   2> 15546 T49 C3 P22421 REQ /replication {file=_9.fdx&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15550 T29 C4 P22410 /update {wt=javabin&version=2} {add=[71 (1415768040100855808)]} 0 2
[junit4:junit4]   2> 15551 T49 C3 P22421 REQ /replication {file=_9_SimpleText_0.pst&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15553 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15554 T47 C3 P22421 /update {wt=javabin&version=2} {add=[10063 (1415768040099807232)]} 0 7
[junit4:junit4]   2> 15556 T49 C3 P22421 REQ /replication {file=_9_MockFixedIntBlock_0.pos&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=0 
[junit4:junit4]   2> 15558 T57 C2 P22439 REQ /update {distrib.from=http://127.0.0.1:22421/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15558 T46 C3 P22421 /update {wt=javabin&version=2} {add=[71 (1415768040106098688)]} 0 5
[junit4:junit4]   2> 15560 T29 C4 P22410 /update {wt=javabin&version=2} {add=[10064 (1415768040111341568)]} 0 2
[junit4:junit4]   2> 15563 T49 C3 P22421 REQ /replication {file=_8_nrm.cfs&command=filecontent&checksum=true&generation=2&qt=/replication&wt=filestream} status=0 QTime=1 
[jun

[...truncated too long message...]

77 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 22421
[junit4:junit4]   2> 39277 T17 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=863145246
[junit4:junit4]   2> 39278 T17 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6d2c0f26
[junit4:junit4]   2> 39285 T17 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1132,cumulative_deletesById=557,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 39286 T17 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 39287 T17 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 39287 T17 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 39290 T17 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 39392 T52 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 39392 T17 oaz.ZooKeeper.close Session: 0x13a5d1845510003 closed
[junit4:junit4]   2> 39394 T17 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 39447 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 22439
[junit4:junit4]   2> 39447 T17 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=361938923
[junit4:junit4]   2> 39549 T116 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 39549 T17 oaz.ZooKeeper.close Session: 0x13a5d1845510007 closed
[junit4:junit4]   2> 39551 T17 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 39735 T17 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RecoveryZkTest -Dtests.method=testDistribSearch -Dtests.seed=E26F3CBC79627AFE -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_IE -Dtests.timezone=Greenwich -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   39.9s J1 | RecoveryZkTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: org.apache.solr.common.SolrException: Server at http://127.0.0.1:22439/solr/collection1 returned non ok status:404, message:Can not find: /solr/collection1/select
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([E26F3CBC79627AFE:6389B2A40E3D1AC2]:0)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:372)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
[junit4:junit4]    > 	at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:92)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:695)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 39856 T17 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene40: {range_facet_l=PostingsFormat(name=MockRandom), text=PostingsFormat(name=SimpleText), _version_=PostingsFormat(name=Asserting), rnd_b=MockFixedIntBlock(blockSize=679), intDefault=PostingsFormat(name=Asserting), timestamp=PostingsFormat(name=MockRandom), id=MockFixedIntBlock(blockSize=679), a_t=PostingsFormat(name=MockRandom), range_facet_sl=MockFixedIntBlock(blockSize=679), range_facet_si=PostingsFormat(name=MockRandom), other_tl1=PostingsFormat(name=Asserting), multiDefault=PostingsFormat(name=MockRandom), a_si=PostingsFormat(name=Asserting)}, sim=DefaultSimilarity, locale=en_IE, timezone=Greenwich
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_06 (64-bit)/cpus=16,threads=1,free=51221224,total=111149056
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SuggesterTest, SpellingQueryConverterTest, RecoveryZkTest]
[junit4:junit4] Completed on J1 in 39.93s, 1 test, 1 error <<< FAILURES!

[...truncated 696 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:38: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:180: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1143: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:816: There were test failures: 231 suites, 957 tests, 1 error, 21 ignored (3 assumptions)

Total time: 31 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message