lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@sd-datasolutions.de>
Subject [JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 696 - Failure!
Date Wed, 25 Jul 2012 08:34:13 GMT
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/696/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1

Stack Trace:
java.lang.AssertionError: shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
	at __randomizedtesting.SeedInfo.seed([E333FE12A5D9F8D4:62D5700AD28698E8]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.FullSolrCloudTest.checkShardConsistency(FullSolrCloudTest.java:1150)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:163)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:679)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
	at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)




Build Log:
[...truncated 7688 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] FAILURE 40.9s J1 | SyncSliceTest.testDistribSearch
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([E333FE12A5D9F8D4:62D5700AD28698E8]:0)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.apache.solr.cloud.FullSolrCloudTest.checkShardConsistency(FullSolrCloudTest.java:1150)
[junit4:junit4]    > 	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:163)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:679)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1995)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:818)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:877)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:891)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:825)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:671)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:697)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:736)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:747)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
[junit4:junit4]    > 
[junit4:junit4]   2> 3 T2345 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/./solrtest-SyncSliceTest-1343205200372
[junit4:junit4]   2> 3 T2345 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 4 T2346 oazs.ZooKeeperServer.setTickTime tickTime set to 3000
[junit4:junit4]   2> 4 T2346 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 5 T2346 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 104 T2345 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:46210 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@459ed93e
[junit4:junit4]   2> 105 T2351 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:46210
[junit4:junit4]   2> 105 T2351 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:46210, initiating session
[junit4:junit4]   2> 105 T2347 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:55350
[junit4:junit4]   2> 105 T2347 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:55350
[junit4:junit4]   2> 106 T2349 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 108 T2349 oazs.NIOServerCnxn.finishSessionInit Established session 0x138bd44d1f60000 with negotiated timeout 10000 for client /127.0.0.1:55350
[junit4:junit4]   2> 108 T2351 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:46210, sessionid = 0x138bd44d1f60000, negotiated timeout = 10000
[junit4:junit4]   2> 108 T2352 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@459ed93e name:ZooKeeperConnection Watcher:127.0.0.1:46210 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 109 T2345 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 111 T2350 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x138bd44d1f60000
[junit4:junit4]   2> 111 T2352 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 112 T2347 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:55350 which had sessionid 0x138bd44d1f60000
[junit4:junit4]   2> 111 T2345 oaz.ZooKeeper.close Session: 0x138bd44d1f60000 closed
[junit4:junit4]   2> 112 T2345 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:46210/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7aa3ff25
[junit4:junit4]   2> 113 T2353 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:46210
[junit4:junit4]   2> 113 T2353 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:46210, initiating session
[junit4:junit4]   2> 113 T2347 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:55351
[junit4:junit4]   2> 113 T2347 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:55351
[junit4:junit4]   2> 114 T2349 oazs.NIOServerCnxn.finishSessionInit Established session 0x138bd44d1f60001 with negotiated timeout 10000 for client /127.0.0.1:55351
[junit4:junit4]   2> 114 T2353 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:46210, sessionid = 0x138bd44d1f60001, negotiated timeout = 10000
[junit4:junit4]   2> 114 T2354 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7aa3ff25 name:ZooKeeperConnection Watcher:127.0.0.1:46210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 114 T2345 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 116 T2345 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 119 T2345 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 121 T2345 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 123 T2345 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 127 T2345 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 130 T2345 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 132 T2345 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 134 T2345 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 136 T2345 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 138 T2345 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 140 T2345 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 141 T2345 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 143 T2345 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 145 T2350 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x138bd44d1f60001
[junit4:junit4]   2> 145 T2354 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 145 T2347 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:55351 which had sessionid 0x138bd44d1f60001
[junit4:junit4]   2> 145 T2345 oaz.ZooKeeper.close Session: 0x138bd44d1f60001 closed
[junit4:junit4]   2> 146 T2345 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 148 T2345 oejs.AbstractConnector.doStart Started SocketConnector@0.0.0.0:56346
[junit4:junit4]   2> 148 T2345 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 148 T2345 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 149 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 151 T2345 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 152 T2345 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 152 T2345 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 152 T2345 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 152 T2345 oasc.CoreContainer.<init> New CoreContainer 1983747994
[junit4:junit4]   2> 152 T2345 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 153 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 157 T2345 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 164 T2345 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:46210/solr
[junit4:junit4]   2> 165 T2345 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:46210/solr sessionTimeout=12000 watcher=org.apache.solr.common.cloud.ConnectionManager@45ac3c1b
[junit4:junit4]   2> 165 T2364 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:46210
[junit4:junit4]   2> 166 T2364 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:46210, initiating session
[junit4:junit4]   2> 166 T2347 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:55352
[junit4:junit4]   2> 166 T2347 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:55352
[junit4:junit4]   2> 167 T2349 oazs.NIOServerCnxn.finishSessionInit Established session 0x138bd44d1f60002 with negotiated timeout 12000 for client /127.0.0.1:55352
[junit4:junit4]   2> 167 T2364 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:46210, sessionid = 0x138bd44d1f60002, negotiated timeout = 12000
[junit4:junit4]   2> 167 T2365 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45ac3c1b name:ZooKeeperConnection Watcher:127.0.0.1:46210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 168 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 169 T2345 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 170 T2345 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56346_solr
[junit4:junit4]   2> 171 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:56346_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:56346_solr
[junit4:junit4]   2> 171 T2345 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56346_solr
[junit4:junit4]   2> 173 T2345 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 173 T2345 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 173 T2345 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 174 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 174 T2345 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 174 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 174 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 174 T2345 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 175 T2345 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 175 T2345 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 177 T2345 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 180 T2345 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 182 T2345 oasc.Overseer.<init> Overseer (id=88028296011710466-127.0.0.1:56346_solr-n_0000000000) starting
[junit4:junit4]   2> 182 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60002 type:create cxid:0x19 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 182 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 183 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 184 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 184 T2367 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 184 T2345 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 185 T2345 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 186 T2366 oasc.Overseer$CloudStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 189 T2345 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 189 T2345 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 196 T2345 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 197 T2345 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 198 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 198 T2345 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 198 T2345 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 219 T2345 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 254 T2345 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 255 T2345 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 260 T2345 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 487 T3 oasc.CoreContainer.finalize SEVERE CoreContainer was not shutdown prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=584982234
[junit4:junit4]   2> 601 T2345 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 607 T2345 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 610 T2345 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 618 T2345 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 621 T2345 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 623 T2345 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 624 T2345 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 625 T2345 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1343205200372/control/data/
[junit4:junit4]   2> 625 T2345 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@99b2061
[junit4:junit4]   2> 625 T2345 oasc.SolrCore.getNewIndexDir WARNING New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1343205200372/control/data/index/
[junit4:junit4]   2> 626 T2345 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1343205200372/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 629 T2345 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/control/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@423b5db3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 629 T2345 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 630 T2345 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 631 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 631 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 631 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 632 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 632 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 633 T2345 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 633 T2345 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 633 T2345 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 633 T2345 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 634 T2345 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 634 T2345 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 635 T2345 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 635 T2345 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 635 T2345 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 635 T2345 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 636 T2345 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 636 T2345 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 636 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 637 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 637 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 637 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 638 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 638 T2345 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 638 T2345 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 639 T2345 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 639 T2345 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 641 T2345 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 641 T2345 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 643 T2345 oass.SolrIndexSearcher.<init> Opening Searcher@3fc24380 main
[junit4:junit4]   2> 644 T2345 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 644 T2345 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 644 T2345 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 647 T2345 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 664 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3fc24380 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 667 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60002 type:create cxid:0x4f zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 668 T2345 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 668 T2345 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:56346/solr shardId:control_shard
[junit4:junit4]   2> 669 T2345 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/control_shard/election
[junit4:junit4]   2> 673 T2345 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/control_shard
[junit4:junit4]   2> 675 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60002 type:create cxid:0x66 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 698 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 700 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60002 type:create cxid:0x71 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 704 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 712 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 712 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 726 T2345 oasc.ZkController.register We are http://127.0.0.1:56346/solr/collection1/ and leader is http://127.0.0.1:56346/solr/collection1/
[junit4:junit4]   2> 726 T2345 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56346/solr
[junit4:junit4]   2> 727 T2345 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 728 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 728 T2345 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1
[junit4:junit4]   2> 729 T2345 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 729 T2345 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 732 T2345 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 733 T2345 oejs.AbstractConnector.doStart Started SocketConnector@0.0.0.0:35171
[junit4:junit4]   2> 734 T2345 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 734 T2345 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 735 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 739 T2345 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 739 T2345 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 739 T2345 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 740 T2345 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 740 T2345 oasc.CoreContainer.<init> New CoreContainer 1571495984
[junit4:junit4]   2> 740 T2345 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 741 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 748 T2345 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 758 T2345 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:46210/solr
[junit4:junit4]   2> 759 T2345 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:46210/solr sessionTimeout=12000 watcher=org.apache.solr.common.cloud.ConnectionManager@dce5f5b
[junit4:junit4]   2> 760 T2378 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:46210
[junit4:junit4]   2> 760 T2378 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:46210, initiating session
[junit4:junit4]   2> 760 T2347 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:55360
[junit4:junit4]   2> 761 T2347 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:55360
[junit4:junit4]   2> 761 T2349 oazs.NIOServerCnxn.finishSessionInit Established session 0x138bd44d1f60003 with negotiated timeout 12000 for client /127.0.0.1:55360
[junit4:junit4]   2> 761 T2378 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:46210, sessionid = 0x138bd44d1f60003, negotiated timeout = 12000
[junit4:junit4]   2> 762 T2379 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dce5f5b name:ZooKeeperConnection Watcher:127.0.0.1:46210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 762 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 762 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 763 T2345 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35171_solr
[junit4:junit4]   2> 764 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:35171_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:35171_solr
[junit4:junit4]   2> 764 T2345 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35171_solr
[junit4:junit4]   2> 765 T2365 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 766 T2345 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 766 T2345 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 766 T2345 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 767 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 767 T2345 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 767 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 768 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 768 T2345 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 768 T2345 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 768 T2345 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 773 T2345 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 778 T2345 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 778 T2345 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 779 T2345 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 780 T2345 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 781 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 781 T2345 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 781 T2345 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 808 T2345 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 849 T2345 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 850 T2345 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 854 T2345 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1185 T2345 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1190 T2345 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1193 T2345 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1201 T2345 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1204 T2345 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1208 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1208 T2345 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1209 T2345 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1222 T2345 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty1/
[junit4:junit4]   2> 1223 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 1223 T2379 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 1223 T2345 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@99b2061
[junit4:junit4]   2> 1223 T2345 oasc.SolrCore.getNewIndexDir WARNING New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty1/index/
[junit4:junit4]   2> 1224 T2345 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1226 T2345 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty1/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51a6b686; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1226 T2345 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 1226 T2345 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1227 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1228 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1228 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1228 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1229 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1229 T2345 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1229 T2345 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1230 T2345 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1230 T2345 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1230 T2345 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1231 T2345 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1231 T2345 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1231 T2345 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1231 T2345 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1232 T2345 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1232 T2345 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1232 T2345 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1233 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1233 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1233 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1233 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1234 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1234 T2345 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1234 T2345 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1235 T2345 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1235 T2345 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1236 T2345 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1237 T2345 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1239 T2345 oass.SolrIndexSearcher.<init> Opening Searcher@15f84ff0 main
[junit4:junit4]   2> 1239 T2345 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1240 T2345 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1240 T2345 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1242 T2345 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1256 T2380 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15f84ff0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1725 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1727 T2379 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 1727 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 1759 T2345 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 1760 T2345 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:35171/solr shardId:shard1
[junit4:junit4]   2> 1761 T2345 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 1765 T2345 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 1769 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60003 type:create cxid:0x4b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2026 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 2026 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 2230 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2235 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 2235 T2379 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 2273 T2345 oasc.ZkController.register We are http://127.0.0.1:35171/solr/collection1/ and leader is http://127.0.0.1:35171/solr/collection1/
[junit4:junit4]   2> 2273 T2345 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35171/solr
[junit4:junit4]   2> 2273 T2345 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 2274 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2275 T2345 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1
[junit4:junit4]   2> 2275 T2345 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2276 T2345 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2278 T2345 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 2283 T2345 oejs.AbstractConnector.doStart Started SocketConnector@0.0.0.0:56875
[junit4:junit4]   2> 2283 T2345 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2283 T2345 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2284 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2286 T2345 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2287 T2345 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2287 T2345 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2287 T2345 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2287 T2345 oasc.CoreContainer.<init> New CoreContainer 1837311076
[junit4:junit4]   2> 2288 T2345 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2288 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2293 T2345 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2300 T2345 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:46210/solr
[junit4:junit4]   2> 2301 T2345 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:46210/solr sessionTimeout=12000 watcher=org.apache.solr.common.cloud.ConnectionManager@3cca5967
[junit4:junit4]   2> 2301 T2390 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:46210
[junit4:junit4]   2> 2302 T2390 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:46210, initiating session
[junit4:junit4]   2> 2302 T2347 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:55369
[junit4:junit4]   2> 2302 T2347 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:55369
[junit4:junit4]   2> 2303 T2349 oazs.NIOServerCnxn.finishSessionInit Established session 0x138bd44d1f60004 with negotiated timeout 12000 for client /127.0.0.1:55369
[junit4:junit4]   2> 2303 T2390 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:46210, sessionid = 0x138bd44d1f60004, negotiated timeout = 12000
[junit4:junit4]   2> 2303 T2391 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cca5967 name:ZooKeeperConnection Watcher:127.0.0.1:46210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2303 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2304 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2305 T2345 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56875_solr
[junit4:junit4]   2> 2305 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:56875_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:56875_solr
[junit4:junit4]   2> 2305 T2345 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56875_solr
[junit4:junit4]   2> 2306 T2365 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 2306 T2379 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 2307 T2345 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2307 T2345 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2307 T2345 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2308 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2308 T2345 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2308 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2308 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2309 T2345 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2309 T2345 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2309 T2345 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2312 T2345 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2316 T2345 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2316 T2345 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 2317 T2345 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 2317 T2345 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 2317 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2318 T2345 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2318 T2345 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2337 T2345 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 2394 T2345 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2396 T2345 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2402 T2345 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2737 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2740 T2379 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 2740 T2391 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 2740 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 2750 T2345 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2756 T2345 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2758 T2345 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2766 T2345 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2769 T2345 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2771 T2345 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2772 T2345 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2773 T2345 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/
[junit4:junit4]   2> 2773 T2345 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@99b2061
[junit4:junit4]   2> 2773 T2345 oasc.SolrCore.getNewIndexDir WARNING New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index/
[junit4:junit4]   2> 2774 T2345 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2776 T2345 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9b67318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2776 T2345 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2777 T2345 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2778 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2778 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2778 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2779 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2779 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2779 T2345 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2780 T2345 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2780 T2345 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2780 T2345 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2781 T2345 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2781 T2345 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2781 T2345 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2782 T2345 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2782 T2345 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2782 T2345 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2782 T2345 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2783 T2345 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2783 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2783 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2784 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2784 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2784 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2785 T2345 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2785 T2345 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2785 T2345 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2786 T2345 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2787 T2345 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2788 T2345 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 2789 T2345 oass.SolrIndexSearcher.<init> Opening Searcher@b45e485 main
[junit4:junit4]   2> 2790 T2345 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2790 T2345 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2791 T2345 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2793 T2345 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2809 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b45e485 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3243 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3245 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 3245 T2391 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 3245 T2379 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 3313 T2345 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 3313 T2345 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:56875/solr shardId:shard1
[junit4:junit4]   2> 3315 T2345 oasc.ZkController.register We are http://127.0.0.1:56875/solr/collection1/ and leader is http://127.0.0.1:35171/solr/collection1/
[junit4:junit4]   2> 3315 T2345 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56875/solr
[junit4:junit4]   2> 3316 T2345 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C94 name=collection1 org.apache.solr.core.SolrCore@7915edb0 url=http://127.0.0.1:56875/solr/collection1 node=127.0.0.1:56875_solr
[junit4:junit4]   2> 3317 T2393 C94 P56875 oasc.RecoveryStrategy.run Starting recovery process. recoveringAfterStartup=true
[junit4:junit4]   2> 3317 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3317 T2393 C94 P56875 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 3318 T2345 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3318 T2345 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3318 T2345 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3319 T2393 C94 P56875 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3321 T2345 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3325 T2345 oejs.AbstractConnector.doStart Started SocketConnector@0.0.0.0:44358
[junit4:junit4]   2> 3326 T2345 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3326 T2345 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3326 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3329 T2345 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3329 T2345 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3330 T2345 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3330 T2345 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 3330 T2345 oasc.CoreContainer.<init> New CoreContainer 2006973415
[junit4:junit4]   2> 3330 T2345 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3330 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3336 T2345 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3343 T2345 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:46210/solr
[junit4:junit4]   2> 3343 T2345 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:46210/solr sessionTimeout=12000 watcher=org.apache.solr.common.cloud.ConnectionManager@2853380c
[junit4:junit4]   2> 3344 T2403 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:46210
[junit4:junit4]   2> 3344 T2403 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:46210, initiating session
[junit4:junit4]   2> 3345 T2347 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:55371
[junit4:junit4]   2> 3345 T2347 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:55371
[junit4:junit4]   2> 3345 T2349 oazs.NIOServerCnxn.finishSessionInit Established session 0x138bd44d1f60005 with negotiated timeout 12000 for client /127.0.0.1:55371
[junit4:junit4]   2> 3345 T2403 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:46210, sessionid = 0x138bd44d1f60005, negotiated timeout = 12000
[junit4:junit4]   2> 3346 T2404 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2853380c name:ZooKeeperConnection Watcher:127.0.0.1:46210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3346 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60005 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3347 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60005 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3347 T2345 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44358_solr
[junit4:junit4]   2> 3348 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60005 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:44358_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:44358_solr
[junit4:junit4]   2> 3348 T2345 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44358_solr
[junit4:junit4]   2> 3349 T2365 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 3349 T2391 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 3349 T2379 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 3350 T2345 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3350 T2345 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3351 T2345 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3351 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3351 T2345 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3351 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3351 T2345 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3352 T2345 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3352 T2345 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3352 T2345 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3356 T2345 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3359 T2345 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 3359 T2345 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 3360 T2345 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 3360 T2345 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 3361 T2345 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 3361 T2345 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 3361 T2345 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3380 T2345 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 3392 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 3393 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 3422 T2345 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 3423 T2345 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3427 T2345 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 3727 T2345 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 3732 T2345 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 3735 T2345 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3744 T2345 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3747 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3748 T2345 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3750 T2379 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 3750 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 3750 T2404 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 3750 T2391 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 3751 T2345 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3752 T2345 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3753 T2345 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/
[junit4:junit4]   2> 3753 T2345 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@99b2061
[junit4:junit4]   2> 3787 T2345 oasc.SolrCore.getNewIndexDir WARNING New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/index/
[junit4:junit4]   2> 3788 T2345 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3790 T2345 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c4ebd2c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3790 T2345 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3791 T2345 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 3792 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 3792 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 3793 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 3793 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 3793 T2345 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 3794 T2345 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 3794 T2345 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 3794 T2345 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3794 T2345 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3795 T2345 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 3795 T2345 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 3796 T2345 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3796 T2345 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 3796 T2345 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 3796 T2345 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 3797 T2345 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3797 T2345 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3797 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3798 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3798 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3798 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3798 T2345 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3799 T2345 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3799 T2345 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 3799 T2345 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 3800 T2345 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3801 T2345 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 3802 T2345 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 3804 T2345 oass.SolrIndexSearcher.<init> Opening Searcher@59d2e43e main
[junit4:junit4]   2> 3804 T2345 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3805 T2345 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3805 T2345 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 3808 T2345 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 3824 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@59d2e43e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4253 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4255 T2379 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 4255 T2391 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 4255 T2404 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 4255 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 4335 T2345 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4336 T2345 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:44358/solr shardId:shard1
[junit4:junit4]   2> 4338 T2345 oasc.ZkController.register We are http://127.0.0.1:44358/solr/collection1/ and leader is http://127.0.0.1:35171/solr/collection1/
[junit4:junit4]   2> 4339 T2345 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44358/solr
[junit4:junit4]   2> 4339 T2345 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C95 name=collection1 org.apache.solr.core.SolrCore@6fa3d6dc url=http://127.0.0.1:44358/solr/collection1 node=127.0.0.1:44358_solr
[junit4:junit4]   2> 4340 T2406 C95 P44358 oasc.RecoveryStrategy.run Starting recovery process. recoveringAfterStartup=true
[junit4:junit4]   2> 4341 T2406 C95 P44358 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 4340 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4342 T2406 C95 P44358 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4342 T2345 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1
[junit4:junit4]   2> 4342 T2345 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4342 T2345 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4345 T2345 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:46210/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@2b8eb1e4
[junit4:junit4]   2> 4346 T2407 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:46210
[junit4:junit4]   2> 4346 T2407 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:46210, initiating session
[junit4:junit4]   2> 4346 T2347 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:55374
[junit4:junit4]   2> 4346 T2347 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:55374
[junit4:junit4]   2> 4347 T2349 oazs.NIOServerCnxn.finishSessionInit Established session 0x138bd44d1f60006 with negotiated timeout 10000 for client /127.0.0.1:55374
[junit4:junit4]   2> 4347 T2407 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:46210, sessionid = 0x138bd44d1f60006, negotiated timeout = 10000
[junit4:junit4]   2> 4347 T2408 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b8eb1e4 name:ZooKeeperConnection Watcher:127.0.0.1:46210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4348 T2345 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4349 T2345 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:false
[junit4:junit4]   2> 4349 T2345 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 4352 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4353 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 4354 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4757 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4760 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 4760 T2404 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 4760 T2379 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 4760 T2391 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 4760 T2408 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 5355 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5462 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 5462 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 6356 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6733 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 6734 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 7357 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8149 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 8150 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 8359 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9360 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10095 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 10096 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2>  C94_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:56875_solr, base_url=http://127.0.0.1:56875/solr}
[junit4:junit4]   2> 10325 T2393 C94 P56875 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:35171/solr/collection1/ recoveringAfterStartup=true
[junit4:junit4]   2> 10325 T2393 C94 P56875 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56875/solr START replicas=[http://127.0.0.1:35171/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 10325 T2393 C94 P56875 oasc.RecoveryStrategy.doRecovery Sync Recovery was not successful - trying replication
[junit4:junit4]   2> 10325 T2393 C94 P56875 oasc.RecoveryStrategy.doRecovery Begin buffering updates
[junit4:junit4]   2> 10326 T2393 C94 P56875 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 10326 T2393 C94 P56875 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:35171/solr/collection1/
[junit4:junit4]   2> 10326 T2393 C94 P56875 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C96 name=collection1 org.apache.solr.core.SolrCore@1d2ec81e url=http://127.0.0.1:35171/solr/collection1 node=127.0.0.1:35171_solr
[junit4:junit4]   2> 10332 T2374 C96 P35171 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 10333 T2373 C96 P35171 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=3 
[junit4:junit4]   2> 10335 T2374 C96 P35171 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty1/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51a6b686; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10335 T2374 C96 P35171 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10336 T2374 C96 P35171 oass.SolrIndexSearcher.<init> Opening Searcher@45dbee8d main
[junit4:junit4]   2> 10336 T2374 C96 P35171 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 10337 T2380 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@45dbee8d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10337 T2374 C96 P35171 /update {waitSearcher=true&commit=true&commit_end_point=true&wt=javabin&softCommit=false&version=2} {commit=} 0 5
[junit4:junit4]   2> 10338 T2393 C94 P56875 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 10338 T2393 C94 P56875 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 10340 T2375 C96 P35171 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 10340 T2375 C96 P35171 REQ /replication {command=indexversion&wt=javabin&qt=/replication&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10342 T2393 C94 P56875 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9b67318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10342 T2393 C94 P56875 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10343 T2393 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 10348 T2393 C94 P56875 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9b67318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9b67318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 10349 T2393 C94 P56875 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 10349 T2393 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@26a6cf80 main
[junit4:junit4]   2> 10350 T2393 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 10350 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@26a6cf80 main{StandardDirectoryReader(segments_2:2)}
[junit4:junit4]   2> 10351 T2393 C94 P56875 oasc.RecoveryStrategy.replay No replay needed
[junit4:junit4]   2> 10351 T2393 C94 P56875 oasc.RecoveryStrategy.doRecovery Recovery was successful - registering as Active
[junit4:junit4]   2> 10352 T2393 C94 P56875 oasc.RecoveryStrategy.doRecovery Finished recovery process
[junit4:junit4]   2> 10361 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10782 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10785 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 10785 T2379 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 10786 T2404 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 10786 T2408 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 10786 T2391 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2>  C95_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:44358_solr, base_url=http://127.0.0.1:44358/solr}
[junit4:junit4]   2> 11346 T2406 C95 P44358 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:35171/solr/collection1/ recoveringAfterStartup=true
[junit4:junit4]   2> 11346 T2406 C95 P44358 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44358/solr START replicas=[http://127.0.0.1:35171/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 11347 T2406 C95 P44358 oasc.RecoveryStrategy.doRecovery Sync Recovery was not successful - trying replication
[junit4:junit4]   2> 11347 T2406 C95 P44358 oasc.RecoveryStrategy.doRecovery Begin buffering updates
[junit4:junit4]   2> 11347 T2406 C95 P44358 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 11347 T2406 C95 P44358 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:35171/solr/collection1/
[junit4:junit4]   2> 11347 T2406 C95 P44358 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11349 T2373 C96 P35171 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=1 
[junit4:junit4]   2> 11350 T2376 C96 P35171 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 11352 T2376 C96 P35171 oass.SolrIndexSearcher.<init> Opening Searcher@710a14a4 main
[junit4:junit4]   2> 11352 T2376 C96 P35171 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 11353 T2380 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@710a14a4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11353 T2376 C96 P35171 /update {waitSearcher=true&commit=true&commit_end_point=true&wt=javabin&softCommit=false&version=2} {commit=} 0 3
[junit4:junit4]   2> 11354 T2406 C95 P44358 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 11354 T2406 C95 P44358 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 11355 T2375 C96 P35171 REQ /replication {command=indexversion&wt=javabin&qt=/replication&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11357 T2406 C95 P44358 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c4ebd2c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11357 T2406 C95 P44358 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 11358 T2406 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 11363 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11378 T2406 C95 P44358 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c4ebd2c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c4ebd2c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 11379 T2406 C95 P44358 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 11379 T2406 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@786b8741 main
[junit4:junit4]   2> 11380 T2406 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 11380 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@786b8741 main{StandardDirectoryReader(segments_2:2)}
[junit4:junit4]   2> 11381 T2406 C95 P44358 oasc.RecoveryStrategy.replay No replay needed
[junit4:junit4]   2> 11381 T2406 C95 P44358 oasc.RecoveryStrategy.doRecovery Recovery was successful - registering as Active
[junit4:junit4]   2> 11383 T2406 C95 P44358 oasc.RecoveryStrategy.doRecovery Finished recovery process
[junit4:junit4]   2> 11789 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11798 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 11798 T2379 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 11798 T2408 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 11798 T2404 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 11798 T2391 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 11950 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 11951 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 12364 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C97 name=collection1 org.apache.solr.core.SolrCore@2d49a29a url=http://127.0.0.1:56346/solr/collection1 node=127.0.0.1:56346_solr
[junit4:junit4]   2> 12370 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 12372 T2357 C97 P56346 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/control/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@423b5db3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12373 T2357 C97 P56346 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 12374 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@6337c832 main
[junit4:junit4]   2> 12375 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 12375 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6337c832 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12375 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 5
[junit4:junit4]   2> 12376 T2345 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:46210/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@63387d81
[junit4:junit4]   2> 12377 T2412 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:46210
[junit4:junit4]   2> 12377 T2412 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:46210, initiating session
[junit4:junit4]   2> 12377 T2347 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /127.0.0.1:55389
[junit4:junit4]   2> 12377 T2347 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /127.0.0.1:55389
[junit4:junit4]   2> 12378 T2349 oazs.NIOServerCnxn.finishSessionInit Established session 0x138bd44d1f60007 with negotiated timeout 10000 for client /127.0.0.1:55389
[junit4:junit4]   2> 12378 T2412 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost.localdomain/127.0.0.1:46210, sessionid = 0x138bd44d1f60007, negotiated timeout = 10000
[junit4:junit4]   2> 12378 T2413 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63387d81 name:ZooKeeperConnection Watcher:127.0.0.1:46210/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12379 T2345 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2>  C94_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56875_solr, base_url=http://127.0.0.1:56875/solr}
[junit4:junit4]   2> 12383 T2383 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 12385 T2383 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@18c1b9a4 main
[junit4:junit4]   2> 12385 T2383 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 12385 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@18c1b9a4 main{StandardDirectoryReader(segments_2:2)}
[junit4:junit4]   2> 12387 T2377 C96 P35171 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 12388 T2377 C96 P35171 oass.SolrIndexSearcher.<init> Opening Searcher@183f69f7 main
[junit4:junit4]   2> 12388 T2377 C96 P35171 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 12389 T2380 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@183f69f7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12389 T2377 C96 P35171 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2>  C95_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44358_solr, base_url=http://127.0.0.1:44358/solr}
[junit4:junit4]   2> 12393 T2396 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 12394 T2396 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@1cfac12e main
[junit4:junit4]   2> 12394 T2396 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 12395 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1cfac12e main{StandardDirectoryReader(segments_2:2)}
[junit4:junit4]   2> 12395 T2396 C95 P44358 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 3
[junit4:junit4]   2> 12396 T2383 C94 P56875 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 13
[junit4:junit4]   2> 12396 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12398 T2371 C96 P35171 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 12399 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 12400 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 13107 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 13107 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 14403 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@aa71ee9 realtime
[junit4:junit4]   2> 14404 T2357 C97 P56346 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1408452751284764672)} 0 2
[junit4:junit4]   2> 14407 T2377 C96 P35171 oass.SolrIndexSearcher.<init> Opening Searcher@41349e75 realtime
[junit4:junit4]   2> 14410 T2396 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@51106dfc realtime
[junit4:junit4]   2> 14410 T2396 C95 P44358 REQ /update {_version_=-1408452751288958976&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14412 T2385 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@1233f61e realtime
[junit4:junit4]   2> 14412 T2385 C94 P56875 REQ /update {_version_=-1408452751288958976&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14413 T2377 C96 P35171 REQ /update {update.distrib=TOLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 14413 T2398 C95 P44358 /update {wt=javabin&version=2} {deleteByQuery=*:*} 0 8
[junit4:junit4]   2> 14416 T2357 C97 P56346 /update {wt=javabin&version=2} {add=[0 (1408452751297347584)]} 0 1
[junit4:junit4]   2> 14424 T2396 C95 P44358 REQ /update {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14427 T2385 C94 P56875 REQ /update {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14428 T2377 C96 P35171 REQ /update {update.distrib=TOLEADER&wt=javabin&version=2} status=0 QTime=8 
[junit4:junit4]   2> 14428 T2398 C95 P44358 /update {wt=javabin&version=2} {add=[0]} 0 10
[junit4:junit4]   2> 14431 T2357 C97 P56346 /update {wt=javabin&version=2} {add=[1 (1408452751313076224)]} 0 1
[junit4:junit4]   2> 14437 T2396 C95 P44358 REQ /update {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14440 T2385 C94 P56875 REQ /update {update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 14441 T2377 C96 P35171 REQ /update {update.distrib=TOLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 14441 T2398 C95 P44358 /update {wt=javabin&version=2} {add=[1]} 0 8
[junit4:junit4]   2> 14443 T2357 C97 P56346 /update {wt=javabin&version=2} {add=[2 (1408452751326707712)]} 0 0
[junit4:junit4]   2> 14450 T2396 C95 P44358 REQ /update {update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:56875/solr/collection1/&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14451 T2377 C96 P35171 REQ /update {update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:56875/solr/collection1/&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 14451 T2383 C94 P56875 /update {test.distrib.skip.servers=http://127.0.0.1:56875/solr/collection1/&wt=javabin&version=2} {add=[2]} 0 6
[junit4:junit4]   2> 14454 T2357 C97 P56346 /update {wt=javabin&version=2} {add=[3 (1408452751337193472)]} 0 1
[junit4:junit4]   2> 14455 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 14455 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 14458 T2377 C96 P35171 REQ /update {update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:56875/solr/collection1/&test.distrib.skip.servers=http://127.0.0.1:44358/solr/collection1/&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14459 T2383 C94 P56875 /update {test.distrib.skip.servers=http://127.0.0.1:56875/solr/collection1/&test.distrib.skip.servers=http://127.0.0.1:44358/solr/collection1/&wt=javabin&version=2} {add=[3]} 0 3
[junit4:junit4]   2> 14461 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14478 T2357 C97 P56346 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/control/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@423b5db3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/control/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@423b5db3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[_0.fnm, _0.frq, _0.tim, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0.prx, _0.tip, _0.fdt]
[junit4:junit4]   2> 14479 T2357 C97 P56346 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 14480 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@1a242eef main
[junit4:junit4]   2> 14481 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14481 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a242eef main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4]   2> 14482 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 21
[junit4:junit4]   2> 14484 T2383 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14508 T2383 C94 P56875 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9b67318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9b67318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_0.fnm, _0.frq, _0.tim, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, _0.prx, _0.tip, _0.fdt, segments_3]
[junit4:junit4]   2> 14509 T2383 C94 P56875 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 14510 T2383 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@3410952c main
[junit4:junit4]   2> 14510 T2383 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14511 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3410952c main{StandardDirectoryReader(segments_3:4:nrt _0(5.0):C2)}
[junit4:junit4]   2> 14514 T2377 C96 P35171 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14528 T2377 C96 P35171 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty1/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51a6b686; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty1/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51a6b686; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[_0.fnm, _0.frq, _0.tim, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0.prx, _0.tip, _0.fdt]
[junit4:junit4]   2> 14529 T2377 C96 P35171 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 14530 T2377 C96 P35171 oass.SolrIndexSearcher.<init> Opening Searcher@570e35cf main
[junit4:junit4]   2> 14531 T2377 C96 P35171 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14531 T2380 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@570e35cf main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4]   2> 14531 T2377 C96 P35171 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 17
[junit4:junit4]   2> 14533 T2396 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14548 T2396 C95 P44358 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c4ebd2c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c4ebd2c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_0.fnm, _0.frq, _0.tim, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, _0.prx, _0.tip, _0.fdt, segments_3]
[junit4:junit4]   2> 14549 T2396 C95 P44358 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 14550 T2396 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@244d9eaa main
[junit4:junit4]   2> 14551 T2396 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14551 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@244d9eaa main{StandardDirectoryReader(segments_3:4:nrt _0(5.0):C3)}
[junit4:junit4]   2> 14552 T2396 C95 P44358 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 19
[junit4:junit4]   2> 14552 T2383 C94 P56875 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 68
[junit4:junit4]   2> 14553 T2345 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14556 T2399 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : action=SYNCSHARD&collection=collection1&shard=shard1&wt=javabin&version=2
[junit4:junit4]   2> 14557 T2399 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14559 T2374 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4]   2> 14560 T2374 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35171/solr/collection1/
[junit4:junit4]   2> 14560 T2374 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:35171/solr START replicas=[http://127.0.0.1:56875/solr/collection1/, http://127.0.0.1:44358/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 14561 T2401 C95 P44358 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=0 
[junit4:junit4]   2> 14561 T2386 C94 P56875 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=0 
[junit4:junit4]   2> 14562 T2374 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:35171/solr  Received 4 versions from 127.0.0.1:44358/solr/collection1/
[junit4:junit4]   2> 14562 T2374 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:35171/solr  Our versions are newer. ourLowThreshold=1408452751288958976 otherHigh=1408452751331950592
[junit4:junit4]   2> 14562 T2374 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:35171/solr  Received 3 versions from 127.0.0.1:56875/solr/collection1/
[junit4:junit4]   2> 14562 T2374 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:35171/solr  Our versions are newer. ourLowThreshold=1408452751288958976 otherHigh=1408452751318319104
[junit4:junit4]   2> 14562 T2374 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:35171/solr DONE. sync succeeded
[junit4:junit4]   2> 14563 T2374 oasc.SyncStrategy.syncToMe try and ask http://127.0.0.1:56875/solr/collection1/ to sync
[junit4:junit4]   2> 14563 T2374 oasc.SyncStrategy.syncToMe try and ask http://127.0.0.1:44358/solr/collection1/ to sync
[junit4:junit4]   2> 14565 T2387 C94 P56875 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56875/solr START replicas=[http://127.0.0.1:35171/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 14565 T2400 C95 P44358 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44358/solr START replicas=[http://127.0.0.1:35171/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 14566 T2372 C96 P35171 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=0 
[junit4:junit4]   2> 14566 T2400 C95 P44358 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44358/solr  Received 5 versions from 127.0.0.1:35171/solr/collection1/
[junit4:junit4]   2> 14566 T2400 C95 P44358 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:44358/solr Requesting updates from 127.0.0.1:35171/solr/collection1/n=1 versions=[1408452751342436352]
[junit4:junit4]   2> 14567 T2373 C96 P35171 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=0 
[junit4:junit4]   2> 14567 T2387 C94 P56875 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56875/solr  Received 5 versions from 127.0.0.1:35171/solr/collection1/
[junit4:junit4]   2> 14567 T2387 C94 P56875 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:56875/solr Requesting updates from 127.0.0.1:35171/solr/collection1/n=2 versions=[1408452751342436352, 1408452751331950592]
[junit4:junit4]   2> 14568 T2372 C96 P35171 REQ /get {distrib=false&getUpdates=1408452751342436352&qt=/get&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14570 T2373 C96 P35171 REQ /get {distrib=false&getUpdates=1408452751342436352,1408452751331950592&qt=/get&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 14571 T2400 C95 P44358 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44358/solr DONE. sync succeeded
[junit4:junit4]   2> 14571 T2400 C95 P44358 REQ /get {sync=http://127.0.0.1:35171/solr/collection1/&getVersions=100&distrib=false&qt=/get&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 14572 T2387 C94 P56875 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56875/solr DONE. sync succeeded
[junit4:junit4]   2> 14573 T2387 C94 P56875 REQ /get {sync=http://127.0.0.1:35171/solr/collection1/&getVersions=100&distrib=false&qt=/get&wt=javabin&version=2} status=0 QTime=9 
[junit4:junit4]   2> 14573 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 14574 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14576 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14577 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@7d3beab2 main
[junit4:junit4]   2> 14577 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14578 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7d3beab2 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4]   2> 14578 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 14580 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14608 T2398 C95 P44358 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c4ebd2c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_0.fnm, _0.frq, _0.tim, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, _0.prx, _0.tip, _0.fdt, segments_3]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty3/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c4ebd2c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[_1_nrm.cfs, _0.fnm, _1.frq, _0_nrm.cfs, _1.fnm, _1_nrm.cfe, _1.fdx, _1.prx, _1.fdt, _1.si, _0_nrm.cfe, _0.tim, _0.frq, _0.fdx, _0.si, _1.tip, _0.tip, _0.prx, _1.tim, _0.fdt, segments_4]
[junit4:junit4]   2> 14609 T2398 C95 P44358 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 14610 T2398 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@2828a8ac main
[junit4:junit4]   2> 14611 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14612 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2828a8ac main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 14614 T2377 C96 P35171 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14614 T2377 C96 P35171 oass.SolrIndexSearcher.<init> Opening Searcher@4c3949d1 main
[junit4:junit4]   2> 14615 T2377 C96 P35171 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14615 T2380 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4c3949d1 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4]   2> 14616 T2377 C96 P35171 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 14618 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14638 T2385 C94 P56875 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9b67318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_0.fnm, _0.frq, _0.tim, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, _0.prx, _0.tip, _0.fdt, segments_3]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9b67318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[_1_nrm.cfs, _0.fnm, _1.frq, _0_nrm.cfs, _1.fnm, _1_nrm.cfe, _1.fdx, _1.prx, _1.fdt, _1.si, _0_nrm.cfe, _0.tim, _0.frq, _0.fdx, _0.si, _1.tip, _0.tip, _0.prx, _1.tim, _0.fdt, segments_4]
[junit4:junit4]   2> 14639 T2385 C94 P56875 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 14640 T2385 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@248953f1 main
[junit4:junit4]   2> 14641 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14641 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@248953f1 main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C2 _1(5.0):C2)}
[junit4:junit4]   2> 14641 T2385 C94 P56875 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 23
[junit4:junit4]   2> 14642 T2398 C95 P44358 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 62
[junit4:junit4]   2> 14643 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14645 T2371 C96 P35171 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> 14647 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> 14649 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> 15654 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 15654 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 16658 T2358 C97 P56346 REQ /select {fl=id,score&shard.url=127.0.0.1:56346/solr/collection1/&NOW=1343205217021&start=0&q=*:*&distrib=false&isShard=true&wt=javabin&fsv=true&rows=10&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> 16670 T2358 C97 P56346 REQ /select {shard.url=127.0.0.1:56346/solr/collection1/&NOW=1343205217021&q=*:*&ids=3,2,1,0&distrib=false&isShard=true&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 16671 T2357 C97 P56346 REQ /select {wt=javabin&q=*:*&version=2} status=0 QTime=19 
[junit4:junit4]   2> Control Docs:4
[junit4:junit4]   2> 16672 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> check const of shard1
[junit4:junit4]   2> client0
[junit4:junit4]   2> PROPS:shard=shard1
[junit4:junit4]   2> leader=true
[junit4:junit4]   2> roles=null
[junit4:junit4]   2> state=active
[junit4:junit4]   2> core=collection1
[junit4:junit4]   2> collection=collection1
[junit4:junit4]   2> node_name=127.0.0.1:35171_solr
[junit4:junit4]   2> base_url=http://127.0.0.1:35171/solr
[junit4:junit4]   2> 
[junit4:junit4]   2> 16675 T2371 C96 P35171 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:4
[junit4:junit4]   2> 
[junit4:junit4]   2> client1
[junit4:junit4]   2> PROPS:shard=shard1
[junit4:junit4]   2> roles=null
[junit4:junit4]   2> state=active
[junit4:junit4]   2> core=collection1
[junit4:junit4]   2> collection=collection1
[junit4:junit4]   2> node_name=127.0.0.1:56875_solr
[junit4:junit4]   2> base_url=http://127.0.0.1:56875/solr
[junit4:junit4]   2> 
[junit4:junit4]   2> 16677 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:4
[junit4:junit4]   2> 
[junit4:junit4]   2> client2
[junit4:junit4]   2> PROPS:shard=shard1
[junit4:junit4]   2> roles=null
[junit4:junit4]   2> state=active
[junit4:junit4]   2> core=collection1
[junit4:junit4]   2> collection=collection1
[junit4:junit4]   2> node_name=127.0.0.1:44358_solr
[junit4:junit4]   2> base_url=http://127.0.0.1:44358/solr
[junit4:junit4]   2> 
[junit4:junit4]   2> 16679 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:4
[junit4:junit4]   2> 
[junit4:junit4]   2> 16684 T2388 C94 P56875 REQ /select {fl=id,score&shard.url=127.0.0.1:35171/solr/collection1/|127.0.0.1:56875/solr/collection1/|127.0.0.1:44358/solr/collection1/&NOW=1343205217050&start=0&q=*:*&distrib=false&isShard=true&wt=javabin&fsv=true&rows=10&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 16690 T2388 C94 P56875 REQ /select {shard.url=127.0.0.1:35171/solr/collection1/|127.0.0.1:56875/solr/collection1/|127.0.0.1:44358/solr/collection1/&NOW=1343205217050&q=*:*&ids=3,2,1,0&distrib=false&isShard=true&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 16691 T2398 C95 P44358 REQ /select {wt=javabin&q=*:*&version=2} status=0 QTime=10 
[junit4:junit4]   2> 16697 T2357 C97 P56346 /update {wt=javabin&version=2} {add=[4 (1408452753689149440)]} 0 2
[junit4:junit4]   2> 16713 T2385 C94 P56875 REQ /update {update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:44358/solr/collection1/&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 16714 T2376 C96 P35171 /update {test.distrib.skip.servers=http://127.0.0.1:44358/solr/collection1/&wt=javabin&version=2} {add=[4 (1408452753693343744)]} 0 15
[junit4:junit4]   2> 16714 T2345 oasc.ChaosMonkey.monkeyLog monkey: kill shard! 35171
[junit4:junit4]   2> 16715 T2345 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1571495984
[junit4:junit4]   2> 16721 T2345 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1d2ec81e
[junit4:junit4]   2> 16723 T2345 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 16724 T2345 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=1,adds=1,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=5,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 16738 T2345 C96 P35171 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty1/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51a6b686; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty1/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51a6b686; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[_0.fnm, _0.frq, _0.tim, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0.prx, _0.tip, _0.fdt]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty1/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@51a6b686; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_1_nrm.cfs, _0.fnm, _1.frq, _0_nrm.cfs, _1.fnm, _1_nrm.cfe, _1.fdx, _1.prx, _1.fdt, _1.si, _0_nrm.cfe, _0.tim, _0.frq, _0.fdx, _0.si, _1.tip, _0.tip, _0.prx, _1.tim, segments_3, _0.fdt]
[junit4:junit4]   2> 16739 T2345 C96 P35171 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 16742 T2350 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x138bd44d1f60003
[junit4:junit4]   2> 16745 T2413 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 16745 T2379 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 16745 T2365 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 16745 T2408 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 16745 T2347 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:55360 which had sessionid 0x138bd44d1f60003
[junit4:junit4]   2> 16745 T2404 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 16746 T2345 oaz.ZooKeeper.close Session: 0x138bd44d1f60003 closed
[junit4:junit4]   2> 16746 T2345 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 16747 T2391 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56875/solr/collection1/
[junit4:junit4]   2> 16747 T2391 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56875/solr START replicas=[http://127.0.0.1:44358/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 16758 T2401 C95 P44358 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=0 
[junit4:junit4]   2> 16758 T2391 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56875/solr  Received 5 versions from 127.0.0.1:44358/solr/collection1/
[junit4:junit4]   2> 16759 T2391 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56875/solr  Our versions are newer. ourLowThreshold=1408452751302590464 otherHigh=1408452751331950592
[junit4:junit4]   2> 16759 T2391 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56875/solr DONE. sync succeeded
[junit4:junit4]   2> 16759 T2391 oasc.SyncStrategy.syncToMe try and ask http://127.0.0.1:35171/solr/collection1/ to sync
[junit4:junit4]   2> 16759 T2391 oasc.SyncStrategy.syncToMe try and ask http://127.0.0.1:44358/solr/collection1/ to sync
[junit4:junit4]   2> 16766 T2391 oasc.SyncStrategy.syncToMe Sync failed - asking replica to recover.
[junit4:junit4]   2> 16766 T2391 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16770 T2391 oasc.SyncStrategy.syncToMe Could not tell a replica to recover org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:35171/solr/collection1
[junit4:junit4]   2> 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:407)
[junit4:junit4]   2> 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:182)
[junit4:junit4]   2> 	at org.apache.solr.cloud.SyncStrategy.syncToMe(SyncStrategy.java:237)
[junit4:junit4]   2> 	at org.apache.solr.cloud.SyncStrategy.syncReplicas(SyncStrategy.java:119)
[junit4:junit4]   2> 	at org.apache.solr.cloud.SyncStrategy.sync(SyncStrategy.java:82)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:158)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:151)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:96)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:57)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:125)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
[junit4:junit4]   2> Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://127.0.0.1:35171 refused
[junit4:junit4]   2> 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:158)
[junit4:junit4]   2> 	at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
[junit4:junit4]   2> 	at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
[junit4:junit4]   2> 	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
[junit4:junit4]   2> 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
[junit4:junit4]   2> 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
[junit4:junit4]   2> 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
[junit4:junit4]   2> 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
[junit4:junit4]   2> 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
[junit4:junit4]   2> 	... 11 more
[junit4:junit4]   2> Caused by: java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4]   2> 	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4]   2> 	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4]   2> 	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4]   2> 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4]   2> 	at java.net.Socket.connect(Socket.java:579)
[junit4:junit4]   2> 	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
[junit4:junit4]   2> 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
[junit4:junit4]   2> 	... 19 more
[junit4:junit4]   2> 
[junit4:junit4]   2> 16772 T2391 oasc.RecoveryStrategy.close WARNING Stopping recovery for core collection1 zkNodeName=127.0.0.1:56875_solr_collection1
[junit4:junit4]   2> 16773 T2391 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 16774 T2350 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x138bd44d1f60004 type:create cxid:0x51 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16776 T2391 oascc.ZkStateReader$3.process Updating live nodes
[junit4:junit4]   2> 16798 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 16799 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16802 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 16812 T2366 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16823 T2404 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 16823 T2408 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 16823 T2365 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 16823 T2391 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 16823 T2413 oascc.ZkStateReader$2.process A cluster state change has occurred
[junit4:junit4]   2> 16838 T2357 C97 P56346 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/control/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@423b5db3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[_0.fnm, _0.frq, _0.tim, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0.prx, _0.tip, _0.fdt]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/control/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@423b5db3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_1_nrm.cfs, _0.fnm, _1.frq, _0_nrm.cfs, _1.fnm, _1_nrm.cfe, _1.fdx, _1.prx, _1.fdt, _1.si, _0_nrm.cfe, _0.tim, _0.frq, _0.fdx, _0.si, _1.tip, _0.tip, _0.prx, _1.tim, segments_3, _0.fdt]
[junit4:junit4]   2> 16839 T2357 C97 P56346 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 16841 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@43ae9d34 main
[junit4:junit4]   2> 16841 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 16842 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@43ae9d34 main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 16842 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 41
[junit4:junit4]   2> 16846 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 16847 T2398 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@795ee0b3 main
[junit4:junit4]   2> 16847 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 16848 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@795ee0b3 main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2>  C94_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, leader=true, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56875_solr, base_url=http://127.0.0.1:56875/solr}
[junit4:junit4]   2> 16857 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 16878 T2385 C94 P56875 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9b67318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[_1_nrm.cfs, _0.fnm, _1.frq, _0_nrm.cfs, _1.fnm, _1_nrm.cfe, _1.fdx, _1.prx, _1.fdt, _1.si, _0_nrm.cfe, _0.tim, _0.frq, _0.fdx, _0.si, _1.tip, _0.tip, _0.prx, _1.tim, _0.fdt, segments_4]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1343205200372/jetty2/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9b67318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[_2.si, _1.frq, _1.fnm, _2.tim, _2_nrm.cfe, _2_nrm.cfs, _0.si, _0.prx, _1.tip, _1.tim, _1_nrm.cfs, _0.fnm, _2.prx, _2.fdt, _2.frq, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _1.prx, _1.fdx, _2.tip, _1.fdt, _1.si, _0.frq, _0.tim, _0_nrm.cfe, _0.fdx, segments_5, _0.tip, _0.fdt]
[junit4:junit4]   2> 16879 T2385 C94 P56875 oasc.SolrDeletionPolicy.updateCommits newest commit = 5
[junit4:junit4]   2> 16881 T2385 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@52ee0ffa main
[junit4:junit4]   2> 16882 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 16883 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@52ee0ffa main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 16883 T2385 C94 P56875 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 26
[junit4:junit4]   2> 16884 T2398 C95 P44358 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 38
[junit4:junit4]   2> 16884 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16888 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=1 
[junit4:junit4]   2> 16891 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 16894 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 16897 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 17164 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 17165 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 17745 T2379 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 17746 T2379 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 18899 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 18900 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18902 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18903 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@63ec4771 main
[junit4:junit4]   2> 18903 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18904 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 18905 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 18905 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@63ec4771 main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 18905 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 3
[junit4:junit4]   2> 18908 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18908 T2398 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@aeb640b main
[junit4:junit4]   2> 18909 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18910 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@aeb640b main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 18912 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18912 T2385 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@67f8f57f main
[junit4:junit4]   2> 18913 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18913 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67f8f57f main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 18913 T2385 C94 P56875 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 1
[junit4:junit4]   2> 18914 T2398 C95 P44358 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 6
[junit4:junit4]   2> 18914 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18921 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 18924 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 18926 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 18929 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 20083 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 20084 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 20930 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 20935 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20944 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20944 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@774bf84d main
[junit4:junit4]   2> 20945 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20945 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@774bf84d main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 20945 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 1
[junit4:junit4]   2> 20948 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20948 T2398 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@74d55b78 main
[junit4:junit4]   2> 20948 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20949 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@74d55b78 main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 20960 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20960 T2385 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@78f1f4b4 main
[junit4:junit4]   2> 20961 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20961 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@78f1f4b4 main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 20962 T2385 C94 P56875 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 20962 T2398 C95 P44358 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 15
[junit4:junit4]   2> 20963 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20966 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 20969 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 20972 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=1 
[junit4:junit4]   2> 20974 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 21820 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 21821 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 22974 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 22975 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 22975 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 22976 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22978 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 22979 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@24446f84 main
[junit4:junit4]   2> 22979 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 22980 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@24446f84 main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 22980 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 22983 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 22983 T2398 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@22ce2452 main
[junit4:junit4]   2> 22984 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 22985 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@22ce2452 main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 22987 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 22988 T2385 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@56e217bf main
[junit4:junit4]   2> 22988 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 22989 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@56e217bf main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 22989 T2385 C94 P56875 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 22990 T2398 C95 P44358 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 7
[junit4:junit4]   2> 22991 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23001 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 23004 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 23006 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=1 
[junit4:junit4]   2> 23008 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 24272 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 24273 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 25008 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 25009 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 25011 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 25012 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@6e5b4f83 main
[junit4:junit4]   2> 25012 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 25013 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6e5b4f83 main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 25013 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 25015 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 25016 T2398 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@49ffa5bd main
[junit4:junit4]   2> 25016 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 25016 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@49ffa5bd main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 25018 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 25019 T2385 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@67e5d2fa main
[junit4:junit4]   2> 25019 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 25020 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67e5d2fa main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 25020 T2385 C94 P56875 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 25021 T2398 C95 P44358 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 6
[junit4:junit4]   2> 25021 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 25024 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 25026 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 25028 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 25030 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 26211 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 26211 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 27031 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 27032 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 27034 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 27035 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@41d18dd2 main
[junit4:junit4]   2> 27035 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 27036 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@41d18dd2 main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 27036 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 27038 T2383 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 27039 T2383 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@269c7114 main
[junit4:junit4]   2> 27039 T2383 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 27039 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@269c7114 main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 27042 T2396 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 27042 T2396 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@79acd378 main
[junit4:junit4]   2> 27042 T2396 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 27043 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@79acd378 main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 27043 T2396 C95 P44358 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 27044 T2383 C94 P56875 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 6
[junit4:junit4]   2> 27044 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 27047 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 27049 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 27051 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 27053 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 27694 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 27694 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 28894 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 28895 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 29055 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 29060 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 29064 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 29064 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@709da983 main
[junit4:junit4]   2> 29065 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 29065 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@709da983 main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 29065 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 1
[junit4:junit4]   2> 29068 T2383 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 29068 T2383 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@2f2672f6 main
[junit4:junit4]   2> 29069 T2383 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 29069 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f2672f6 main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 29071 T2396 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 29072 T2396 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@5fe8f84f main
[junit4:junit4]   2> 29072 T2396 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 29073 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5fe8f84f main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 29073 T2396 C95 P44358 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 29074 T2383 C94 P56875 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 7
[junit4:junit4]   2> 29074 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 29077 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 29080 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 29082 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 29084 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 30201 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 30202 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 31085 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 31086 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 31088 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 31089 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@5c4f5ce3 main
[junit4:junit4]   2> 31089 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 31090 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5c4f5ce3 main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 31090 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 31093 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 31094 T2398 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@43a84a80 main
[junit4:junit4]   2> 31094 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 31095 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@43a84a80 main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 31103 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 31104 T2385 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@f430400 main
[junit4:junit4]   2> 31104 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 31108 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f430400 main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 31109 T2385 C94 P56875 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 6
[junit4:junit4]   2> 31109 T2398 C95 P44358 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 16
[junit4:junit4]   2> 31110 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 31113 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 31116 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 31119 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=1 
[junit4:junit4]   2> 31121 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 31626 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 31626 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 32908 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 32909 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 33123 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 33124 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 33127 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 33127 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@22a4118c main
[junit4:junit4]   2> 33127 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 33128 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@22a4118c main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 33128 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 33131 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 33131 T2398 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@3804b93e main
[junit4:junit4]   2> 33132 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 33133 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3804b93e main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 33146 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 33146 T2385 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@5d87a803 main
[junit4:junit4]   2> 33147 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 33148 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5d87a803 main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 33148 T2385 C94 P56875 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 33150 T2398 C95 P44358 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 19
[junit4:junit4]   2> 33151 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 33156 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 33159 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 33161 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 33164 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 34535 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 34535 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 35165 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 35166 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 35168 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 35169 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@6f9a7f51 main
[junit4:junit4]   2> 35169 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 35170 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f9a7f51 main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 35170 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 35173 T2383 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 35173 T2383 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@1dae4edc main
[junit4:junit4]   2> 35174 T2383 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 35174 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1dae4edc main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 35184 T2396 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 35185 T2396 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@44cf70e4 main
[junit4:junit4]   2> 35185 T2396 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 35186 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44cf70e4 main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 35186 T2396 C95 P44358 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 35187 T2383 C94 P56875 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 14
[junit4:junit4]   2> 35188 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 35191 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 35193 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 35199 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 35202 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 35669 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 35670 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 37098 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 37098 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 37203 T2345 oasc.AbstractDistributedZkTestCase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):360
[junit4:junit4]   2> 37204 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 37207 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 37207 T2357 C97 P56346 oass.SolrIndexSearcher.<init> Opening Searcher@72e82be main
[junit4:junit4]   2> 37207 T2357 C97 P56346 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 37208 T2368 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72e82be main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C4 _1(5.0):C1)}
[junit4:junit4]   2> 37208 T2357 C97 P56346 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 2
[junit4:junit4]   2> 37210 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 37211 T2398 C95 P44358 oass.SolrIndexSearcher.<init> Opening Searcher@782df646 main
[junit4:junit4]   2> 37211 T2398 C95 P44358 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 37212 T2405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@782df646 main{StandardDirectoryReader(segments_4:6:nrt _0(5.0):C3 _1(5.0):C1)}
[junit4:junit4]   2> 37214 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 37214 T2385 C94 P56875 oass.SolrIndexSearcher.<init> Opening Searcher@5dd03245 main
[junit4:junit4]   2> 37215 T2385 C94 P56875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 37215 T2392 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5dd03245 main{StandardDirectoryReader(segments_5:8:nrt _0(5.0):C2 _1(5.0):C2 _2(5.0):C1)}
[junit4:junit4]   2> 37215 T2385 C94 P56875 /update {waitSearcher=true&commit=true&wt=javabin&commit_end_point=true&softCommit=false&version=2} {commit=} 0 1
[junit4:junit4]   2> 37216 T2398 C95 P44358 /update {waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 6
[junit4:junit4]   2> 37217 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 37220 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=1 
[junit4:junit4]   2> 37222 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 37224 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=1 
[junit4:junit4]   2> 37225 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 37230 T2358 C97 P56346 REQ /select {fl=id,score&shard.url=127.0.0.1:56346/solr/collection1/&NOW=1343205237596&start=0&q=*:*&distrib=false&isShard=true&wt=javabin&fsv=true&rows=10&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 37232 T2358 C97 P56346 REQ /select {shard.url=127.0.0.1:56346/solr/collection1/&NOW=1343205237596&q=*:*&ids=3,2,1,0,4&distrib=false&isShard=true&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 37233 T2357 C97 P56346 REQ /select {wt=javabin&q=*:*&version=2} status=0 QTime=6 
[junit4:junit4]   2> Control Docs:5
[junit4:junit4]   2> 37233 T2345 oascc.ZkStateReader.updateCloudState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> check const of shard1
[junit4:junit4]   2> client0
[junit4:junit4]   2> PROPS:shard=shard1
[junit4:junit4]   2> roles=null
[junit4:junit4]   2> state=active
[junit4:junit4]   2> core=collection1
[junit4:junit4]   2> collection=collection1
[junit4:junit4]   2> node_name=127.0.0.1:35171_solr
[junit4:junit4]   2> base_url=http://127.0.0.1:35171/solr
[junit4:junit4]   2> 
[junit4:junit4]   2> error contacting client: Server refused connection at: http://localhost:35171/solr/collection1
[junit4:junit4]   2> 
[junit4:junit4]   2> client1
[junit4:junit4]   2> PROPS:shard=shard1
[junit4:junit4]   2> roles=null
[junit4:junit4]   2> leader=true
[junit4:junit4]   2> state=active
[junit4:junit4]   2> core=collection1
[junit4:junit4]   2> collection=collection1
[junit4:junit4]   2> node_name=127.0.0.1:56875_solr
[junit4:junit4]   2> base_url=http://127.0.0.1:56875/solr
[junit4:junit4]   2> 
[junit4:junit4]   2> 37236 T2384 C94 P56875 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:5
[junit4:junit4]   2> 
[junit4:junit4]   2> client2
[junit4:junit4]   2> PROPS:shard=shard1
[junit4:junit4]   2> roles=null
[junit4:junit4]   2> state=active
[junit4:junit4]   2> core=collection1
[junit4:junit4]   2> collection=collection1
[junit4:junit4]   2> node_name=127.0.0.1:44358_solr
[junit4:junit4]   2> base_url=http://127.0.0.1:44358/solr
[junit4:junit4]   2> 
[junit4:junit4]   2> 37238 T2397 C95 P44358 REQ /select {distrib=false&wt=javabin&q=*:*&version=2} hits=4 status=0 QTime=0 
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:4
[junit4:junit4]   2> 
[junit4:junit4]   2> ######shard1 is not consistent.  Got 5 from org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3dlastClient and got 4 from http://127.0.0.1:44358/solr/collection1
[junit4:junit4]   2> 37240 T2384 C94 P56875 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=5 status=0 QTime=0 
[junit4:junit4]   2> 37242 T2397 C95 P44358 REQ /select {fl=id,_version_&sort=id+asc&q=*:*&distrib=false&wt=javabin&rows=1000&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: {numFound=5,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}, SolrDocument{id=4, _version_=1408452753693343744}]}
[junit4:junit4]   2> ######org.apache.solr.client.solrj.impl.HttpSolrServer@3cdeda9d: {numFound=4,start=0,docs=[SolrDocument{id=0, _version_=1408452751302590464}, SolrDocument{id=1, _version_=1408452751318319104}, SolrDocument{id=2, _version_=1408452751331950592}, SolrDocument{id=3, _version_=1408452751342436352}]}
[junit4:junit4]   2> ###### sizes=5,4
[junit4:junit4]   2> ###### Only in org.apache.solr.client.solrj.impl.HttpSolrServer@28e70f3d: [{id=4, _version_=1408452753693343744}]
[junit4:junit4]   2> 37242 T2350 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x138bd44d1f60007
[junit4:junit4]   2> 37244 T2413 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 37244 T2347 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:55389 which had sessionid 0x138bd44d1f60007
[junit4:junit4]   2> 37244 T2345 oaz.ZooKeeper.close Session: 0x138bd44d1f60007 closed
[junit4:junit4]   2> 37244 T2350 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x138bd44d1f60006
[junit4:junit4]   2> 37245 T2347 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:55374 which had sessionid 0x138bd44d1f60006
[junit4:junit4]   2> 37245 T2345 oaz.ZooKeeper.close Session: 0x138bd44d1f60006 closed
[junit4:junit4]   2> 37245 T2349 oazs.SyncRequestProcessor.run SyncRequestProcessor exited!
[junit4:junit4]   2> 37245 T2350 oazs.PrepRequestProcessor.run PrepRequestProcessor exited loop!
[junit4:junit4]   2> 37245 T2408 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 37245 T2345 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4]   2> 37247 T2345 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:55371 which had sessionid 0x138bd44d1f60005
[junit4:junit4]   2> 37247 T2403 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x138bd44d1f60005, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 37247 T2345 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:55352 which had sessionid 0x138bd44d1f60002
[junit4:junit4]   2> 37248 T2364 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x138bd44d1f60002, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 37248 T2345 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:55369 which had sessionid 0x138bd44d1f60004
[junit4:junit4]   2> 37248 T2390 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x138bd44d1f60004, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 37248 T2347 oazs.NIOServerCnxn$Factory.run NIOServerCnxn factory exited run method
[junit4:junit4]   2> 37249 T2345 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4]   2> 37249 T2345 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 56346
[junit4:junit4]   2> 37249 T2345 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1983747994
[junit4:junit4]   2> 37249 T2345 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2d49a29a
[junit4:junit4]   2> 37250 T2345 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 37251 T2345 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=14,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=5,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 37348 T2404 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2853380c name:ZooKeeperConnection Watcher:127.0.0.1:46210/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4]   2> 37348 T2365 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45ac3c1b name:ZooKeeperConnection Watcher:127.0.0.1:46210/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4]   2> 37348 T2367 oasc.OverseerCollectionProcessor.run WARNING Overseer cannot talk to ZK
[junit4:junit4]   2> 37348 T2365 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 37348 T2345 oaz.ZooKeeper.close Session: 0x138bd44d1f60002 closed
[junit4:junit4]   2> 37348 T2391 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cca5967 name:ZooKeeperConnection Watcher:127.0.0.1:46210/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4]   2> 37349 T2345 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 37379 T2366 oasc.Overseer$CloudStateUpdater.amILeader WARNING  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:927)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:289)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:286)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:67)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:286)
[junit4:junit4]   2> 	at org.apache.solr.cloud.Overseer$CloudStateUpdater.amILeader(Overseer.java:187)
[junit4:junit4]   2> 	at org.apache.solr.cloud.Overseer$CloudStateUpdater.run(Overseer.java:113)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 37379 T2366 oasc.Overseer$CloudStateUpdater.amILeader According to ZK I (id=88028296011710466-127.0.0.1:56346_solr-n_0000000000) am no longer a leader.
[junit4:junit4]   2> 37401 T2345 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 35171
[junit4:junit4]   2> 37402 T2345 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 56875
[junit4:junit4]   2> 37402 T2345 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1837311076
[junit4:junit4]   2> 37402 T2345 oasc.RecoveryStrategy.close WARNING Stopping recovery for core collection1 zkNodeName=127.0.0.1:56875_solr_collection1
[junit4:junit4]   2> 37402 T2345 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7915edb0
[junit4:junit4]   2> 37403 T2345 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 37404 T2345 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=15,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=5,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 37405 T2345 oasc.RecoveryStrategy.close WARNING Stopping recovery for core collection1 zkNodeName=127.0.0.1:56875_solr_collection1
[junit4:junit4]   2> 38314 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 38314 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 38776 T2403 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:46210
[junit4:junit4]   2> 38776 T2403 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd44d1f60005 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 39238 T2390 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:46210
[junit4:junit4]   2> 39339 T2391 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 39339 T2345 oaz.ZooKeeper.close Session: 0x138bd44d1f60004 closed
[junit4:junit4]   2> 39340 T2345 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 39391 T2345 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 44358
[junit4:junit4]   2> 39391 T2345 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=2006973415
[junit4:junit4]   2> 39392 T2345 oasc.RecoveryStrategy.close WARNING Stopping recovery for core collection1 zkNodeName=127.0.0.1:44358_solr_collection1
[junit4:junit4]   2> 39392 T2345 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6fa3d6dc
[junit4:junit4]   2> 39394 T2345 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 39394 T2345 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=15,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=4,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 39395 T2345 oasc.RecoveryStrategy.close WARNING Stopping recovery for core collection1 zkNodeName=127.0.0.1:44358_solr_collection1
[junit4:junit4]   2> 39631 T2348 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4]   2> 40253 T1844 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:47811
[junit4:junit4]   2> 40254 T1844 oaz.ClientCnxn$SendThread.run WARNING Session 0x138bd41b8530010 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4:junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4]   2> 
[junit4:junit4]   2> 40334 T2403 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost.localdomain/127.0.0.1:46210
[junit4:junit4]   2> 40435 T2404 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 40435 T2345 oaz.ZooKeeper.close Session: 0x138bd44d1f60005 closed
[junit4:junit4]   2> 40436 T2345 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 40501 T2345 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=E333FE12A5D9F8D4 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=bg -Dtests.timezone=Pacific/Port_Moresby -Dtests.file.encoding=UTF-8
[junit4:junit4]   2>
[junit4:junit4]    > (@AfterClass output)
[junit4:junit4]   2> 40939 T2345 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Asserting, sim=DefaultSimilarity, locale=bg, timezone=Pacific/Port_Moresby
[junit4:junit4]   2> NOTE: Linux 3.2.0-26-generic amd64/Oracle Corporation 1.7.0_05 (64-bit)/cpus=8,threads=2,free=331788808,total=503316480
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestPersianNormalizationFilterFactory, TestFastLRUCache, CloudStateTest, TestBeiderMorseFilterFactory, TestSolrDeletionPolicy2, TestShingleFilterFactory, TestCoreContainer, TestReplicationHandler, TestPropInjectDefaults, TestDocumentBuilder, ZkControllerTest, SortByFunctionTest, TestNumberUtils, TestGreekStemFilterFactory, TestCzechStemFilterFactory, TestBinaryResponseWriter, SolrCoreTest, TestGermanNormalizationFilterFactory, StandardRequestHandlerTest, SoftAutoCommitTest, AlternateDirectoryTest, CloudStateUpdateTest, TestTrie, TestHindiFilters, TestIndonesianStemFilterFactory, TestLuceneMatchVersion, TestQueryTypes, TestHungarianLightStemFilterFactory, QueryEqualityTest, TestTurkishLowerCaseFilterFactory, TestWikipediaTokenizerFactory, TestWriterPerf, DistributedTermsComponentTest, SpellCheckCollatorTest, NoCacheHeaderTest, BasicZkTest, TestMultiWordSynonyms, TestXIncludeConfig, IndexSchemaRuntimeFieldTest, TestDocSet, DebugComponentTest, UniqFieldsUpdateProcessorFactoryTest, TestJmxMonitoredMap, TestCollationField, TestPatternReplaceFilterFactory, TestGalicianMinimalStemFilterFactory, LeaderElectionTest, TestPortugueseLightStemFilterFactory, FieldAnalysisRequestHandlerTest, TestSort, TestJapaneseTokenizerFactory, TestStopFilterFactory, TestGroupingSearch, XsltUpdateRequestHandlerTest, SpellPossibilityIteratorTest, TestSolrXMLSerializer, TimeZoneUtilsTest, CommonGramsFilterFactoryTest, SampleTest, TermVectorComponentTest, TestPseudoReturnFields, DistanceFunctionTest, TestDefaultSimilarityFactory, LengthFilterTest, TestReverseStringFilterFactory, TestUtils, OverseerTest, HighlighterConfigTest, PeerSyncTest, TestIndexSearcher, TestNorwegianMinimalStemFilterFactory, SolrPluginUtilsTest, FullSolrCloudTest, DistributedSpellCheckComponentTest, TestRealTimeGet, TestDistributedSearch, AutoCommitTest, TestPhoneticFilterFactory, ZkSolrClientTest, TestFaceting, TestJoin, CopyFieldTest, TestQueryUtils, UpdateParamsTest, ShowFileRequestHandlerTest, NotRequiredUniqueKeyTest, TestSolrDeletionPolicy1, TestPropInject, TestLFUCache, TestQuerySenderNoQuery, NumericFieldsTest, TestJapaneseKatakanaStemFilterFactory, TestJapaneseBaseFormFilterFactory, TestConfig, TestBM25SimilarityFactory, TestPluginEnable, TestJapaneseReadingFormFilterFactory, TestIBSimilarityFactory, URLClassifyProcessorTest, PrimUtilsTest, TestNGramFilters, TestUAX29URLEmailTokenizerFactory, ResourceLoaderTest, TestPortugueseMinimalStemFilterFactory, DateFieldTest, TestSuggestSpellingConverter, TestBadConfig, TestKeywordMarkerFilterFactory, CommonGramsQueryFilterFactoryTest, RAMDirectoryFactoryTest, SnowballPorterFilterFactoryTest, TestGermanStemFilterFactory, TestKeepFilterFactory, TestPorterStemFilterFactory, TestCJKBigramFilterFactory, TestGermanMinimalStemFilterFactory, CircularListTest, TestRussianLightStemFilterFactory, SyncSliceTest]
[junit4:junit4]   2> 
[junit4:junit4] Completed on J1 in 40.95s, 1 test, 1 failure <<< FAILURES!

[...truncated 11 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/build.xml:29: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/solr/build.xml:144: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/lucene/module-build.xml:62: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/lucene/common-build.xml:1094: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux-Java7-64/checkout/lucene/common-build.xml:789: There were test failures: 297 suites, 1055 tests, 1 failure, 5 ignored

Total time: 18 minutes 44 seconds
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure




Mime
View raw message