lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b79) - Build # 4817 - Failure!
Date Fri, 22 Mar 2013 02:49:16 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/4817/
Java: 32bit/jdk1.8.0-ea-b79 -client -XX:+UseParallelGC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest

Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=806, name=coreLoadExecutor-457-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.

Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=806, name=coreLoadExecutor-457-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
	at java.lang.Thread.getStackTrace(Thread.java:1569)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:524)
	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:991)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:883)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:875)
	at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:37)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:327)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:244)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:458)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:746)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:619)
	at org.apache.solr.core.CoreContainer.createFromZk(CoreContainer.java:872)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:955)
	at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:579)
	at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:574)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.RAMDirectory@3e2c64 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1588dba)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest

Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=808, name=RecoveryThread, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.

Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=808, name=RecoveryThread, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
	at java.lang.Thread.getStackTrace(Thread.java:1569)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:524)
	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:991)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:883)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:875)
	at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:37)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:327)
	at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:383)
	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:281)
	at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:153)
	at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409)
	at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:223)
Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ca4702 lockFactory=org.apache.lucene.store.NativeFSLockFactory@41fbf7)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


REGRESSION:  org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch

Error Message:
Still found shard2 in collection test_unload_shard_and_collection

Stack Trace:
java.lang.AssertionError: Still found shard2 in collection test_unload_shard_and_collection
	at __randomizedtesting.SeedInfo.seed([8A99804DB37337B1:B7F0E55C42C578D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:124)
	at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:487)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9099 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.UnloadDistributedZkTest
[junit4:junit4]   2> 0 T368 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /iw_/
[junit4:junit4]   2> 5 T368 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076
[junit4:junit4]   2> 6 T368 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T369 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T369 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T369 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 10 T369 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T369 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 12 T369 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-1363919394076/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 107 T368 oasc.ZkTestServer.run start zk server on port:53441
[junit4:junit4]   2> 108 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@824ddb
[junit4:junit4]   2> 109 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 113 T374 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 114 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36563
[junit4:junit4]   2> 115 T374 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 115 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36563
[junit4:junit4]   2> 116 T372 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 228 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230000 with negotiated timeout 10000 for client /127.0.0.1:36563
[junit4:junit4]   2> 228 T374 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230000, negotiated timeout = 10000
[junit4:junit4]   2> 228 T375 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@824ddb name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 229 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 229 T368 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 232 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230000
[junit4:junit4]   2> 233 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36563 which had sessionid 0x13d8fee55230000
[junit4:junit4]   2> 233 T375 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 233 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230000 closed
[junit4:junit4]   2> 234 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@45b73b
[junit4:junit4]   2> 235 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 235 T376 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 235 T376 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 235 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36564
[junit4:junit4]   2> 236 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36564
[junit4:junit4]   2> 237 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230001 with negotiated timeout 10000 for client /127.0.0.1:36564
[junit4:junit4]   2> 237 T376 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230001, negotiated timeout = 10000
[junit4:junit4]   2> 237 T377 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45b73b name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 237 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 238 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 241 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 244 T368 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 246 T368 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 249 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 250 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 255 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 256 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 263 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 264 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 269 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 269 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 273 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 274 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 277 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 278 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 280 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 281 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 284 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 284 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 287 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 287 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 290 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230001
[junit4:junit4]   2> 290 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36564 which had sessionid 0x13d8fee55230001
[junit4:junit4]   2> 290 T377 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 290 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230001 closed
[junit4:junit4]   2> 369 T368 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 379 T368 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47038
[junit4:junit4]   2> 380 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 380 T368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 381 T368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1363919394361
[junit4:junit4]   2> 381 T368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1363919394361/solr.xml
[junit4:junit4]   2> 382 T368 oasc.CoreContainer.<init> New CoreContainer 16251349
[junit4:junit4]   2> 382 T368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1363919394361/'
[junit4:junit4]   2> 382 T368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1363919394361/'
[junit4:junit4]   2> 406 T368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 407 T368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 407 T368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 408 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 408 T368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 409 T368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 409 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 410 T368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 410 T368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 411 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 423 T368 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 444 T368 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53441/solr
[junit4:junit4]   2> 444 T368 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 445 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@cf64e1
[junit4:junit4]   2> 446 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 446 T387 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 447 T387 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 447 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36565
[junit4:junit4]   2> 447 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36565
[junit4:junit4]   2> 448 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230002 with negotiated timeout 20000 for client /127.0.0.1:36565
[junit4:junit4]   2> 448 T387 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230002, negotiated timeout = 20000
[junit4:junit4]   2> 448 T388 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cf64e1 name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 449 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 450 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230002
[junit4:junit4]   2> 450 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36565 which had sessionid 0x13d8fee55230002
[junit4:junit4]   2> 450 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230002 closed
[junit4:junit4]   2> 451 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 452 T388 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 455 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7d1b06
[junit4:junit4]   2> 456 T389 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 456 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 457 T389 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 457 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36566
[junit4:junit4]   2> 458 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36566
[junit4:junit4]   2> 458 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230003 with negotiated timeout 20000 for client /127.0.0.1:36566
[junit4:junit4]   2> 458 T389 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230003, negotiated timeout = 20000
[junit4:junit4]   2> 459 T390 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d1b06 name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 459 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 461 T368 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 464 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:create cxid:0x7 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 465 T368 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 469 T368 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 470 T368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47038_iw_
[junit4:junit4]   2> 471 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:delete cxid:0x12 zxid:0x1e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:47038_iw_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:47038_iw_
[junit4:junit4]   2> 472 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47038_iw_
[junit4:junit4]   2> 478 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 489 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:delete cxid:0x20 zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 490 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 493 T368 oasc.Overseer.start Overseer (id=89385821410623491-127.0.0.1:47038_iw_-n_0000000000) starting
[junit4:junit4]   2> 494 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:create cxid:0x25 zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 495 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:create cxid:0x26 zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 497 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:create cxid:0x28 zxid:0x27 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 499 T368 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 504 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:create cxid:0x2e zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 506 T392 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 506 T368 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 511 T368 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 514 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 517 T391 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 520 T368 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 520 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 521 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 525 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 527 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@77d658
[junit4:junit4]   2> 528 T394 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 528 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 529 T394 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 529 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36567
[junit4:junit4]   2> 529 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36567
[junit4:junit4]   2> 530 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230004 with negotiated timeout 10000 for client /127.0.0.1:36567
[junit4:junit4]   2> 530 T394 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230004, negotiated timeout = 10000
[junit4:junit4]   2> 531 T395 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77d658 name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 531 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 532 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 536 T368 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 626 T368 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 630 T368 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47486
[junit4:junit4]   2> 630 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 631 T368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 631 T368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607
[junit4:junit4]   2> 631 T368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/solr.xml
[junit4:junit4]   2> 632 T368 oasc.CoreContainer.<init> New CoreContainer 20573689
[junit4:junit4]   2> 632 T368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/'
[junit4:junit4]   2> 633 T368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/'
[junit4:junit4]   2> 657 T368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 658 T368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 658 T368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 659 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 659 T368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 659 T368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 660 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 660 T368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 661 T368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 661 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 670 T368 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 686 T368 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53441/solr
[junit4:junit4]   2> 687 T368 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 687 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@cf4eaf
[junit4:junit4]   2> 688 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 688 T405 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 689 T405 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 689 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36568
[junit4:junit4]   2> 690 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36568
[junit4:junit4]   2> 690 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230005 with negotiated timeout 20000 for client /127.0.0.1:36568
[junit4:junit4]   2> 690 T405 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230005, negotiated timeout = 20000
[junit4:junit4]   2> 691 T406 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cf4eaf name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 691 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 692 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230005
[junit4:junit4]   2> 693 T406 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 693 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36568 which had sessionid 0x13d8fee55230005
[junit4:junit4]   2> 693 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230005 closed
[junit4:junit4]   2> 693 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 696 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@11e371e
[junit4:junit4]   2> 698 T407 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 698 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 698 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36569
[junit4:junit4]   2> 698 T407 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 699 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36569
[junit4:junit4]   2> 699 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230006 with negotiated timeout 20000 for client /127.0.0.1:36569
[junit4:junit4]   2> 699 T407 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230006, negotiated timeout = 20000
[junit4:junit4]   2> 700 T408 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11e371e name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 700 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 701 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:create cxid:0x1 zxid:0x30 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 702 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:create cxid:0x3 zxid:0x31 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 704 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1707 T368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47486_iw_
[junit4:junit4]   2> 1708 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:delete cxid:0xd zxid:0x32 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:47486_iw_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:47486_iw_
[junit4:junit4]   2> 1709 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47486_iw_
[junit4:junit4]   2> 1712 T390 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1712 T395 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1712 T408 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1718 T368 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 1719 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1719 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1816 T368 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 1818 T368 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51183
[junit4:junit4]   2> 1819 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1819 T368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1820 T368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792
[junit4:junit4]   2> 1820 T368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/solr.xml
[junit4:junit4]   2> 1821 T368 oasc.CoreContainer.<init> New CoreContainer 29144622
[junit4:junit4]   2> 1822 T368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/'
[junit4:junit4]   2> 1822 T368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/'
[junit4:junit4]   2> 1852 T368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 1853 T368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1853 T368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 1854 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1854 T368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1855 T368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1855 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1856 T368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1856 T368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1857 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 1868 T368 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1889 T368 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53441/solr
[junit4:junit4]   2> 1890 T368 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1890 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1403692
[junit4:junit4]   2> 1892 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1892 T419 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1894 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36571
[junit4:junit4]   2> 1894 T419 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 1894 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36571
[junit4:junit4]   2> 1895 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230007 with negotiated timeout 20000 for client /127.0.0.1:36571
[junit4:junit4]   2> 1895 T419 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230007, negotiated timeout = 20000
[junit4:junit4]   2> 1896 T420 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1403692 name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1896 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1898 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230007
[junit4:junit4]   2> 1899 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36571 which had sessionid 0x13d8fee55230007
[junit4:junit4]   2> 1900 T420 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 1900 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230007 closed
[junit4:junit4]   2> 1900 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1905 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@85d6f2
[junit4:junit4]   2> 1907 T421 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1907 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1907 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36572
[junit4:junit4]   2> 1907 T421 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 1908 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36572
[junit4:junit4]   2> 1909 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230008 with negotiated timeout 20000 for client /127.0.0.1:36572
[junit4:junit4]   2> 1909 T421 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230008, negotiated timeout = 20000
[junit4:junit4]   2> 1909 T422 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@85d6f2 name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1910 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1911 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230008 type:create cxid:0x1 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1912 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230008 type:create cxid:0x3 zxid:0x39 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1918 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2922 T368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51183_iw_
[junit4:junit4]   2> 2923 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230008 type:delete cxid:0xd zxid:0x3a txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:51183_iw_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:51183_iw_
[junit4:junit4]   2> 2925 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51183_iw_
[junit4:junit4]   2> 2932 T390 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 2933 T395 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 2934 T408 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 2935 T422 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 2946 T368 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 2948 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2949 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3045 T368 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3048 T368 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55443
[junit4:junit4]   2> 3048 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3049 T368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3049 T368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026
[junit4:junit4]   2> 3050 T368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/solr.xml
[junit4:junit4]   2> 3050 T368 oasc.CoreContainer.<init> New CoreContainer 18644155
[junit4:junit4]   2> 3050 T368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/'
[junit4:junit4]   2> 3051 T368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/'
[junit4:junit4]   2> 3073 T368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 3073 T368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3074 T368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 3074 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3074 T368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3075 T368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3075 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3076 T368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3076 T368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3076 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 3085 T368 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3101 T368 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53441/solr
[junit4:junit4]   2> 3102 T368 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 3102 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@17fc368
[junit4:junit4]   2> 3103 T433 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3108 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3120 T433 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 3120 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36573
[junit4:junit4]   2> 3121 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36573
[junit4:junit4]   2> 3121 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230009 with negotiated timeout 20000 for client /127.0.0.1:36573
[junit4:junit4]   2> 3122 T433 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230009, negotiated timeout = 20000
[junit4:junit4]   2> 3122 T434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17fc368 name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3123 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3124 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230009
[junit4:junit4]   2> 3125 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36573 which had sessionid 0x13d8fee55230009
[junit4:junit4]   2> 3125 T434 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 3125 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230009 closed
[junit4:junit4]   2> 3126 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 3130 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@939704
[junit4:junit4]   2> 3131 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3132 T435 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3132 T435 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 3132 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36574
[junit4:junit4]   2> 3133 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36574
[junit4:junit4]   2> 3133 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee5523000a with negotiated timeout 20000 for client /127.0.0.1:36574
[junit4:junit4]   2> 3133 T435 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee5523000a, negotiated timeout = 20000
[junit4:junit4]   2> 3134 T436 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@939704 name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3134 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3135 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000a type:create cxid:0x1 zxid:0x40 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3136 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000a type:create cxid:0x3 zxid:0x41 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3141 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4144 T368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55443_iw_
[junit4:junit4]   2> 4145 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000a type:delete cxid:0xd zxid:0x42 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:55443_iw_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:55443_iw_
[junit4:junit4]   2> 4146 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55443_iw_
[junit4:junit4]   2> 4149 T395 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 4149 T390 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 4149 T408 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 4149 T422 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 4149 T436 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 4154 T368 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 4155 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4155 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4234 T368 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4236 T368 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59877
[junit4:junit4]   2> 4237 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4237 T368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4237 T368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1363919398229
[junit4:junit4]   2> 4238 T368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1363919398229/solr.xml
[junit4:junit4]   2> 4238 T368 oasc.CoreContainer.<init> New CoreContainer 31507358
[junit4:junit4]   2> 4239 T368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1363919398229/'
[junit4:junit4]   2> 4239 T368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1363919398229/'
[junit4:junit4]   2> 4262 T368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 4263 T368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4263 T368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4264 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4264 T368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4265 T368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4265 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4266 T368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4266 T368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4266 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4274 T368 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4289 T368 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53441/solr
[junit4:junit4]   2> 4289 T368 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4290 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3846a3
[junit4:junit4]   2> 4291 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4292 T447 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4293 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36575
[junit4:junit4]   2> 4293 T447 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 4293 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36575
[junit4:junit4]   2> 4294 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee5523000b with negotiated timeout 20000 for client /127.0.0.1:36575
[junit4:junit4]   2> 4294 T447 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee5523000b, negotiated timeout = 20000
[junit4:junit4]   2> 4295 T448 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3846a3 name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4295 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4297 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee5523000b
[junit4:junit4]   2> 4297 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36575 which had sessionid 0x13d8fee5523000b
[junit4:junit4]   2> 4297 T368 oaz.ZooKeeper.close Session: 0x13d8fee5523000b closed
[junit4:junit4]   2> 4297 T448 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4298 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 4302 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@19e97be
[junit4:junit4]   2> 4303 T449 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4303 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4304 T449 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 4304 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36576
[junit4:junit4]   2> 4305 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36576
[junit4:junit4]   2> 4306 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee5523000c with negotiated timeout 20000 for client /127.0.0.1:36576
[junit4:junit4]   2> 4306 T449 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee5523000c, negotiated timeout = 20000
[junit4:junit4]   2> 4306 T450 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19e97be name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4306 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4308 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000c type:create cxid:0x1 zxid:0x48 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4309 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000c type:create cxid:0x3 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4311 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5315 T368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59877_iw_
[junit4:junit4]   2> 5315 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000c type:delete cxid:0xd zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:59877_iw_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:59877_iw_
[junit4:junit4]   2> 5316 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59877_iw_
[junit4:junit4]   2> 5319 T390 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 5319 T450 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 5319 T436 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 5319 T395 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 5319 T422 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 5319 T408 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 5324 T368 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4]   2> 5325 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5325 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 5330 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 5341 T404 oasc.CoreContainer.create Creating SolrCore 'unloadcollection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/unloadcollection1
[junit4:junit4]   2> 5342 T404 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection
[junit4:junit4]   2> 5342 T404 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:unloadcollection
[junit4:junit4]   2> 5343 T404 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 5343 T404 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 5344 T404 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection
[junit4:junit4]   2> 5346 T404 oasc.ZkController.readConfigName Load collection config from:/collections/unloadcollection
[junit4:junit4]   2> 5347 T404 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/unloadcollection1/'
[junit4:junit4]   2> 5379 T404 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 5424 T404 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5425 T404 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5430 T404 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5921 T404 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5928 T404 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5932 T404 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5944 T404 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5948 T404 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5952 T404 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5953 T404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 5954 T404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 5954 T404 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5955 T404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 5956 T404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 5956 T404 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 5956 T404 oasc.SolrCore.<init> [unloadcollection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/unloadcollection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n/
[junit4:junit4]   2> 5957 T404 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@143f408
[junit4:junit4]   2> 5958 T404 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 5958 T404 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n
[junit4:junit4]   2> 5959 T404 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n/index/
[junit4:junit4]   2> 5959 T404 oasc.SolrCore.initIndex WARNING [unloadcollection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5960 T404 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n/index
[junit4:junit4]   2> 5961 T404 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5962 T404 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5974 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5974 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5975 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5975 T404 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5976 T404 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5976 T404 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5976 T404 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5977 T404 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5977 T404 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5980 T404 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5983 T404 oass.SolrIndexSearcher.<init> Opening Searcher@1d77c39 main
[junit4:junit4]   2> 5984 T404 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n/tlog
[junit4:junit4]   2> 5987 T404 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5987 T404 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5991 T452 oasc.SolrCore.registerSearcher [unloadcollection1] Registered new searcher Searcher@1d77c39 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5992 T404 oasc.ZkController.publish publishing core=unloadcollection1 state=down
[junit4:junit4]   2> 6527 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6528 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "core":"unloadcollection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "collection":"unloadcollection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:47486_iw_",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:47486/iw_",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down"}
[junit4:junit4]   2> 6528 T391 oasc.Overseer$ClusterStateUpdater.createCollection Create collection unloadcollection with numShards 1
[junit4:junit4]   2> 6537 T391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6543 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 6543 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 6543 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 6543 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 6543 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 6543 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 6993 T404 oasc.CoreContainer.registerCore registering core: unloadcollection1
[junit4:junit4]   2> 6994 T404 oasc.ZkController.register Register replica - core:unloadcollection1 address:http://127.0.0.1:47486/iw_ collection:unloadcollection shard:shard1
[junit4:junit4]   2> 7001 T404 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 7012 T404 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection/leader_elect/shard1/election
[junit4:junit4]   2> 7021 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:delete cxid:0x51 zxid:0x57 txntype:-1 reqpath:n/a Error Path:/solr/collections/unloadcollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/unloadcollection/leaders
[junit4:junit4]   2> 7022 T404 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7023 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:create cxid:0x52 zxid:0x58 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7026 T404 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7026 T404 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7026 T404 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47486/iw_/unloadcollection1/
[junit4:junit4]   2> 7027 T404 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7027 T404 oasc.SyncStrategy.syncToMe http://127.0.0.1:47486/iw_/unloadcollection1/ has no replicas
[junit4:junit4]   2> 7028 T404 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47486/iw_/unloadcollection1/
[junit4:junit4]   2> 7028 T404 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection/leaders/shard1
[junit4:junit4]   2> 7032 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:create cxid:0x5d zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8048 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8054 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 8054 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 8054 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 8054 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 8054 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 8054 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 8086 T404 oasc.ZkController.register We are http://127.0.0.1:47486/iw_/unloadcollection1/ and leader is http://127.0.0.1:47486/iw_/unloadcollection1/
[junit4:junit4]   2> 8088 T404 oasc.ZkController.register No LogReplay needed for core=unloadcollection1 baseURL=http://127.0.0.1:47486/iw_
[junit4:junit4]   2> 8088 T404 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8089 T404 oasc.ZkController.publish publishing core=unloadcollection1 state=active
[junit4:junit4]   2> 8090 T404 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8091 T404 oasc.CoreContainer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/solr.xml
[junit4:junit4]   2> 8099 T404 oasc.SolrXMLSerializer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/solr.xml
[junit4:junit4]   2> 8100 T404 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {action=CREATE&version=2&numShards=1&collection=unloadcollection&name=unloadcollection1&wt=javabin&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n} status=0 QTime=2760 
[junit4:junit4]   2> 8100 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 8102 T368 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1550e0e
[junit4:junit4]   2> 8103 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8103 T453 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8104 T453 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session
[junit4:junit4]   2> 8104 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36578
[junit4:junit4]   2> 8104 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36578
[junit4:junit4]   2> 8106 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee5523000d with negotiated timeout 10000 for client /127.0.0.1:36578
[junit4:junit4]   2> 8106 T453 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee5523000d, negotiated timeout = 10000
[junit4:junit4]   2> 8106 T454 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1550e0e name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8107 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8108 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 8110 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8111 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8116 T415 oasc.CoreContainer.create Creating SolrCore 'unloadcollection2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/unloadcollection2
[junit4:junit4]   2> 8117 T415 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection
[junit4:junit4]   2> 8117 T415 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 8118 T415 oasc.ZkController.readConfigName Load collection config from:/collections/unloadcollection
[junit4:junit4]   2> 8119 T415 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/unloadcollection2/'
[junit4:junit4]   2> 8162 T415 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 8209 T415 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 8210 T415 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 8215 T415 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8690 T415 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8700 T415 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8705 T415 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8721 T415 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8725 T415 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8729 T415 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8731 T415 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 8732 T415 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 8732 T415 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8733 T415 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 8733 T415 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 8734 T415 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 8734 T415 oasc.SolrCore.<init> [unloadcollection2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/unloadcollection2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/
[junit4:junit4]   2> 8734 T415 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@143f408
[junit4:junit4]   2> 8735 T415 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 8735 T415 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n
[junit4:junit4]   2> 8736 T415 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/index/
[junit4:junit4]   2> 8736 T415 oasc.SolrCore.initIndex WARNING [unloadcollection2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8736 T415 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/index
[junit4:junit4]   2> 8738 T415 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@c1f890 lockFactory=org.apache.lucene.store.NativeFSLockFactory@32ce9e; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8738 T415 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8740 T415 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8740 T415 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8740 T415 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8741 T415 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8741 T415 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8742 T415 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8742 T415 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8743 T415 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8743 T415 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8746 T415 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8750 T415 oass.SolrIndexSearcher.<init> Opening Searcher@140cc01 main
[junit4:junit4]   2> 8750 T415 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/tlog
[junit4:junit4]   2> 8751 T415 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8751 T415 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8755 T455 oasc.SolrCore.registerSearcher [unloadcollection2] Registered new searcher Searcher@140cc01 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8756 T415 oasc.ZkController.publish publishing core=unloadcollection2 state=down
[junit4:junit4]   2> 8756 T415 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9560 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9561 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "core":"unloadcollection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "collection":"unloadcollection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:47486_iw_",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:47486_iw__unloadcollection1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:47486/iw_",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active"}
[junit4:junit4]   2> 9565 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "core":"unloadcollection2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "collection":"unloadcollection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:51183_iw_",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:51183/iw_",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down"}
[junit4:junit4]   2> 9565 T391 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 9566 T391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 9569 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 9569 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 9569 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 9569 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 9569 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 9569 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 9569 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 9758 T415 oasc.CoreContainer.registerCore registering core: unloadcollection2
[junit4:junit4]   2> 9758 T415 oasc.ZkController.register Register replica - core:unloadcollection2 address:http://127.0.0.1:51183/iw_ collection:unloadcollection shard:shard1
[junit4:junit4]   2> 9764 T415 oasc.ZkController.register We are http://127.0.0.1:51183/iw_/unloadcollection2/ and leader is http://127.0.0.1:47486/iw_/unloadcollection1/
[junit4:junit4]   2> 9764 T415 oasc.ZkController.register No LogReplay needed for core=unloadcollection2 baseURL=http://127.0.0.1:51183/iw_
[junit4:junit4]   2> 9765 T415 oasc.ZkController.checkRecovery Core needs to recover:unloadcollection2
[junit4:junit4]   2> 9765 T415 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C25 name=unloadcollection2 org.apache.solr.core.SolrCore@1467e93 url=http://127.0.0.1:51183/iw_/unloadcollection2 node=127.0.0.1:51183_iw_ C25_STATE=coll:unloadcollection core:unloadcollection2 props:{core=unloadcollection2, collection=unloadcollection, node_name=127.0.0.1:51183_iw_, base_url=http://127.0.0.1:51183/iw_, state=down}
[junit4:junit4]   2> 9773 T456 C25 P51183 oasc.RecoveryStrategy.run Starting recovery process.  core=unloadcollection2 recoveringAfterStartup=true
[junit4:junit4]   2> 9774 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 9775 T456 C25 P51183 oasc.ZkController.publish publishing core=unloadcollection2 state=recovering
[junit4:junit4]   2> 9775 T456 C25 P51183 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9773 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9777 T456 C25 P51183 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9777 T415 oasc.CoreContainer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/solr.xml
[junit4:junit4]   2> 9789 T415 oasc.SolrXMLSerializer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/solr.xml
[junit4:junit4]   2> 9790 T415 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {action=CREATE&version=2&collection=unloadcollection&name=unloadcollection2&wt=javabin&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n} status=0 QTime=1674 
[junit4:junit4]   2> 9792 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9793 T368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: unloadcollection failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 9793 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10795 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11076 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11077 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "core":"unloadcollection2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "collection":"unloadcollection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:51183_iw_",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:51183_iw__unloadcollection2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:51183/iw_",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering"}
[junit4:junit4]   2> 11087 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 11088 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 11088 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 11088 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 11088 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 11088 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 11091 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 11784 T401 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:51183_iw__unloadcollection2&core=unloadcollection1&nodeName=127.0.0.1:51183_iw_&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&wt=javabin&state=recovering} status=0 QTime=2000 
[junit4:junit4]   2> 11798 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12799 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C25_STATE=coll:unloadcollection core:unloadcollection2 props:{core=unloadcollection2, collection=unloadcollection, node_name=127.0.0.1:51183_iw_, shard=shard1, base_url=http://127.0.0.1:51183/iw_, state=recovering}
[junit4:junit4]   2> 13785 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:47486/iw_/unloadcollection1/ core=unloadcollection2 - recoveringAfterStartup=true
[junit4:junit4]   2> 13791 T456 C25 P51183 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 13798 T456 C25 P51183 oasu.PeerSync.sync PeerSync: core=unloadcollection2 url=http://127.0.0.1:51183/iw_ START replicas=[http://127.0.0.1:47486/iw_/unloadcollection1/] nUpdates=100
[junit4:junit4]   2> 13799 T456 C25 P51183 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 13807 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=unloadcollection2
[junit4:junit4]   2> 13807 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=unloadcollection2
[junit4:junit4]   2> 13807 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=unloadcollection2
[junit4:junit4]   2> 13808 T456 C25 P51183 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 13808 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13808 T456 C25 P51183 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:47486/iw_/unloadcollection1/. core=unloadcollection2
[junit4:junit4]   2> 13808 T456 C25 P51183 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C26 name=unloadcollection1 org.apache.solr.core.SolrCore@38adab url=http://127.0.0.1:47486/iw_/unloadcollection1 node=127.0.0.1:47486_iw_ C26_STATE=coll:unloadcollection core:unloadcollection1 props:{core=unloadcollection1, collection=unloadcollection, node_name=127.0.0.1:47486_iw_, shard=shard1, base_url=http://127.0.0.1:47486/iw_, state=active, leader=true}
[junit4:junit4]   2> 13813 T402 C26 P47486 REQ /get {getVersions=100&version=2&distrib=false&qt=/get&wt=javabin} status=0 QTime=1 
[junit4:junit4]   2> 13822 T404 C26 P47486 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 13824 T404 C26 P47486 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13824 T404 C26 P47486 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13825 T404 C26 P47486 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13825 T404 C26 P47486 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13826 T404 C26 P47486 oass.SolrIndexSearcher.<init> Opening Searcher@122430c realtime
[junit4:junit4]   2> 13826 T404 C26 P47486 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 13827 T404 C26 P47486 /update {waitSearcher=true&softCommit=false&version=2&wt=javabin&commit_end_point=true&commit=true&openSearcher=false} {commit=} 0 11
[junit4:junit4]   2> 13857 T456 C25 P51183 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 13857 T456 C25 P51183 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 13864 T456 C25 P51183 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 13873 T398 C26 P47486 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 13874 T398 C26 P47486 REQ /replication {version=2&qt=/replication&command=indexversion&wt=javabin} status=0 QTime=8 
[junit4:junit4]   2> 13876 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 13876 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 13877 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 13879 T401 C26 P47486 REQ /replication {version=2&qt=/replication&command=filelist&wt=javabin&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 13880 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 13881 T456 C25 P51183 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/index.20130322033007950
[junit4:junit4]   2> 13882 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Starting download to BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6b345c lockFactory=org.apache.lucene.store.NativeFSLockFactory@92b992) fullCopy=false
[junit4:junit4]   2> 13890 T402 C26 P47486 REQ /replication {checksum=true&qt=/replication&command=filecontent&wt=filestream&file=segments_2&generation=2} status=0 QTime=3 
[junit4:junit4]   2> 13897 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 13899 T456 C25 P51183 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 13899 T456 C25 P51183 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=unloadcollection2
[junit4:junit4]   2> 13900 T456 C25 P51183 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@c1f890 lockFactory=org.apache.lucene.store.NativeFSLockFactory@32ce9e; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@c1f890 lockFactory=org.apache.lucene.store.NativeFSLockFactory@32ce9e; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13901 T456 C25 P51183 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13901 T456 C25 P51183 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 13901 T456 C25 P51183 oass.SolrIndexSearcher.<init> Opening Searcher@1d529e3 main
[junit4:junit4]   2> 13902 T455 oasc.SolrCore.registerSearcher [unloadcollection2] Registered new searcher Searcher@1d529e3 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 13903 T456 C25 P51183 oasc.RecoveryStrategy.replay No replay needed. core=unloadcollection2
[junit4:junit4]   2> 13903 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=unloadcollection2
[junit4:junit4]   2> 13903 T456 C25 P51183 oasc.ZkController.publish publishing core=unloadcollection2 state=active
[junit4:junit4]   2> 13904 T456 C25 P51183 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13905 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=unloadcollection2
[junit4:junit4]   2> 14097 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14098 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "core":"unloadcollection2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "collection":"unloadcollection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:51183_iw_",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:51183_iw__unloadcollection2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:51183/iw_",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active"}
[junit4:junit4]   2> 14102 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14102 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14102 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14102 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14102 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14103 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14102 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14813 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14814 T368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: unloadcollection
[junit4:junit4]   2> 14814 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14819 T431 oasc.CoreContainer.create Creating SolrCore 'unloadcollection3' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/unloadcollection3
[junit4:junit4]   2> 14820 T431 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection
[junit4:junit4]   2> 14820 T431 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 14821 T431 oasc.ZkController.readConfigName Load collection config from:/collections/unloadcollection
[junit4:junit4]   2> 14822 T431 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/unloadcollection3/'
[junit4:junit4]   2> 14862 T431 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 14917 T431 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 14918 T431 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 14925 T431 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 15445 T431 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 15452 T431 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 15457 T431 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 15468 T431 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15472 T431 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15475 T431 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15477 T431 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 15477 T431 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 15478 T431 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15479 T431 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 15479 T431 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 15480 T431 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 15480 T431 oasc.SolrCore.<init> [unloadcollection3] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/unloadcollection3/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/
[junit4:junit4]   2> 15480 T431 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@143f408
[junit4:junit4]   2> 15481 T431 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 15482 T431 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n
[junit4:junit4]   2> 15482 T431 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/index/
[junit4:junit4]   2> 15482 T431 oasc.SolrCore.initIndex WARNING [unloadcollection3] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 15483 T431 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/index
[junit4:junit4]   2> 15485 T431 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@16b45d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6af33; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15485 T431 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15487 T431 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 15487 T431 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 15488 T431 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 15488 T431 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 15489 T431 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 15489 T431 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 15490 T431 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 15490 T431 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 15491 T431 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 15494 T431 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 15497 T431 oass.SolrIndexSearcher.<init> Opening Searcher@15b6a1e main
[junit4:junit4]   2> 15498 T431 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/tlog
[junit4:junit4]   2> 15498 T431 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 15499 T431 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 15504 T459 oasc.SolrCore.registerSearcher [unloadcollection3] Registered new searcher Searcher@15b6a1e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 15504 T431 oasc.ZkController.publish publishing core=unloadcollection3 state=down
[junit4:junit4]   2> 15505 T431 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15609 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15611 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "core":"unloadcollection3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "collection":"unloadcollection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:55443_iw_",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55443/iw_",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down"}
[junit4:junit4]   2> 15611 T391 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 15611 T391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 15618 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15618 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15618 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15618 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15618 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15619 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15618 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16506 T431 oasc.CoreContainer.registerCore registering core: unloadcollection3
[junit4:junit4]   2> 16507 T431 oasc.ZkController.register Register replica - core:unloadcollection3 address:http://127.0.0.1:55443/iw_ collection:unloadcollection shard:shard1
[junit4:junit4]   2> 16512 T431 oasc.ZkController.register We are http://127.0.0.1:55443/iw_/unloadcollection3/ and leader is http://127.0.0.1:47486/iw_/unloadcollection1/
[junit4:junit4]   2> 16513 T431 oasc.ZkController.register No LogReplay needed for core=unloadcollection3 baseURL=http://127.0.0.1:55443/iw_
[junit4:junit4]   2> 16514 T431 oasc.ZkController.checkRecovery Core needs to recover:unloadcollection3
[junit4:junit4]   2> 16515 T431 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C27 name=unloadcollection3 org.apache.solr.core.SolrCore@12d0b21 url=http://127.0.0.1:55443/iw_/unloadcollection3 node=127.0.0.1:55443_iw_ C27_STATE=coll:unloadcollection core:unloadcollection3 props:{core=unloadcollection3, collection=unloadcollection, node_name=127.0.0.1:55443_iw_, base_url=http://127.0.0.1:55443/iw_, state=down}
[junit4:junit4]   2> 16515 T460 C27 P55443 oasc.RecoveryStrategy.run Starting recovery process.  core=unloadcollection3 recoveringAfterStartup=true
[junit4:junit4]   2> 16516 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 16517 T460 C27 P55443 oasc.ZkController.publish publishing core=unloadcollection3 state=recovering
[junit4:junit4]   2> 16517 T460 C27 P55443 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16517 T431 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16518 T460 C27 P55443 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16518 T431 oasc.CoreContainer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/solr.xml
[junit4:junit4]   2> 16528 T431 oasc.SolrXMLSerializer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/solr.xml
[junit4:junit4]   2> 16529 T431 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {action=CREATE&version=2&collection=unloadcollection&name=unloadcollection3&wt=javabin&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n} status=0 QTime=1710 
[junit4:junit4]   2> 16530 T368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: unloadcollection failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 16531 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17124 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17125 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "core":"unloadcollection3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "collection":"unloadcollection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:55443_iw_",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:55443_iw__unloadcollection3",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55443/iw_",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering"}
[junit4:junit4]   2> 17128 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17128 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17128 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17128 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17128 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17128 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17128 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17525 T398 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:55443_iw__unloadcollection3&core=unloadcollection1&nodeName=127.0.0.1:55443_iw_&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&wt=javabin&state=recovering} status=0 QTime=1000 
[junit4:junit4]   2> 17532 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18534 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C27_STATE=coll:unloadcollection core:unloadcollection3 props:{core=unloadcollection3, collection=unloadcollection, node_name=127.0.0.1:55443_iw_, shard=shard1, base_url=http://127.0.0.1:55443/iw_, state=recovering}
[junit4:junit4]   2> 19526 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:47486/iw_/unloadcollection1/ core=unloadcollection3 - recoveringAfterStartup=true
[junit4:junit4]   2> 19526 T460 C27 P55443 oasu.PeerSync.sync PeerSync: core=unloadcollection3 url=http://127.0.0.1:55443/iw_ START replicas=[http://127.0.0.1:47486/iw_/unloadcollection1/] nUpdates=100
[junit4:junit4]   2> 19527 T460 C27 P55443 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 19527 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=unloadcollection3
[junit4:junit4]   2> 19527 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=unloadcollection3
[junit4:junit4]   2> 19527 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=unloadcollection3
[junit4:junit4]   2> 19527 T460 C27 P55443 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 19527 T460 C27 P55443 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:47486/iw_/unloadcollection1/. core=unloadcollection3
[junit4:junit4]   2> 19527 T460 C27 P55443 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19529 T404 C26 P47486 REQ /get {getVersions=100&version=2&distrib=false&qt=/get&wt=javabin} status=0 QTime=0 
[junit4:junit4]   2> 19533 T401 C26 P47486 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 19534 T401 C26 P47486 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 19534 T401 C26 P47486 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 19535 T401 C26 P47486 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 19535 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19535 T401 C26 P47486 /update {waitSearcher=true&softCommit=false&version=2&wt=javabin&commit_end_point=true&commit=true&openSearcher=false} {commit=} 0 2
[junit4:junit4]   2> 19536 T460 C27 P55443 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 19536 T460 C27 P55443 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 19539 T398 C26 P47486 REQ /replication {version=2&qt=/replication&command=indexversion&wt=javabin} status=0 QTime=0 
[junit4:junit4]   2> 19539 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4]   2> 19539 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 19540 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 19542 T404 C26 P47486 REQ /replication {version=2&qt=/replication&command=filelist&wt=javabin&generation=3} status=0 QTime=1 
[junit4:junit4]   2> 19542 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 19543 T460 C27 P55443 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/index.20130322033013613
[junit4:junit4]   2> 19544 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Starting download to BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@7a682e lockFactory=org.apache.lucene.store.NativeFSLockFactory@8b6574) fullCopy=false
[junit4:junit4]   2> 19546 T401 C26 P47486 REQ /replication {checksum=true&qt=/replication&command=filecontent&wt=filestream&file=segments_3&generation=3} status=0 QTime=0 
[junit4:junit4]   2> 19548 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 19549 T460 C27 P55443 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 19549 T460 C27 P55443 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=unloadcollection3
[junit4:junit4]   2> 19550 T460 C27 P55443 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@16b45d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6af33; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@16b45d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6af33; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 19551 T460 C27 P55443 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 19552 T460 C27 P55443 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 19552 T460 C27 P55443 oass.SolrIndexSearcher.<init> Opening Searcher@1cc236d main
[junit4:junit4]   2> 19553 T459 oasc.SolrCore.registerSearcher [unloadcollection3] Registered new searcher Searcher@1cc236d main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4]   2> 19553 T460 C27 P55443 oasc.RecoveryStrategy.replay No replay needed. core=unloadcollection3
[junit4:junit4]   2> 19554 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=unloadcollection3
[junit4:junit4]   2> 19554 T460 C27 P55443 oasc.ZkController.publish publishing core=unloadcollection3 state=active
[junit4:junit4]   2> 19554 T460 C27 P55443 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19556 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=unloadcollection3
[junit4:junit4]   2> 20136 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20137 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "core":"unloadcollection3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "collection":"unloadcollection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:55443_iw_",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:55443_iw__unloadcollection3",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55443/iw_",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active"}
[junit4:junit4]   2> 20141 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20141 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20141 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20141 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20141 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20141 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20141 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20537 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20537 T368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: unloadcollection
[junit4:junit4]   2> 20538 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2>  C27_STATE=coll:unloadcollection core:unloadcollection3 props:{core=unloadcollection3, collection=unloadcollection, node_name=127.0.0.1:55443_iw_, shard=shard1, base_url=http://127.0.0.1:55443/iw_, state=active}
[junit4:junit4]   2> 20577 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[20 (1430173164116115456)]} 0 2
[junit4:junit4]   2>  C25_STATE=coll:unloadcollection core:unloadcollection2 props:{core=unloadcollection2, collection=unloadcollection, node_name=127.0.0.1:51183_iw_, shard=shard1, base_url=http://127.0.0.1:51183/iw_, state=active}
[junit4:junit4]   2> 20584 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[20 (1430173164116115456)]} 0 2
[junit4:junit4]   2> 20585 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[20 (1430173164116115456)]} 0 24
[junit4:junit4]   2> 20586 T430 C27 P55443 /update {version=2&wt=javabin} {add=[20]} 0 38
[junit4:junit4]   2> 20597 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[21 (1430173164147572736)]} 0 3
[junit4:junit4]   2> 20597 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[21 (1430173164147572736)]} 0 3
[junit4:junit4]   2> 20603 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[21 (1430173164147572736)]} 0 13
[junit4:junit4]   2> 20603 T429 C27 P55443 /update {version=2&wt=javabin} {add=[21]} 0 15
[junit4:junit4]   2> 20613 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[22 (1430173164165398528)]} 0 1
[junit4:junit4]   2> 20615 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[22 (1430173164165398528)]} 0 3
[junit4:junit4]   2> 20616 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[22 (1430173164165398528)]} 0 8
[junit4:junit4]   2> 20616 T431 C27 P55443 /update {version=2&wt=javabin} {add=[22]} 0 10
[junit4:junit4]   2> 20626 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[23 (1430173164179030016)]} 0 2
[junit4:junit4]   2> 20626 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[23 (1430173164179030016)]} 0 2
[junit4:junit4]   2> 20627 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[23 (1430173164179030016)]} 0 6
[junit4:junit4]   2> 20627 T432 C27 P55443 /update {version=2&wt=javabin} {add=[23]} 0 8
[junit4:junit4]   2> 20636 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[24 (1430173164190564352)]} 0 1
[junit4:junit4]   2> 20637 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[24 (1430173164190564352)]} 0 2
[junit4:junit4]   2> 20638 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[24 (1430173164190564352)]} 0 5
[junit4:junit4]   2> 20638 T430 C27 P55443 /update {version=2&wt=javabin} {add=[24]} 0 8
[junit4:junit4]   2> 20648 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[25 (1430173164202098688)]} 0 2
[junit4:junit4]   2> 20648 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[25 (1430173164202098688)]} 0 2
[junit4:junit4]   2> 20649 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[25 (1430173164202098688)]} 0 6
[junit4:junit4]   2> 20649 T429 C27 P55443 /update {version=2&wt=javabin} {add=[25]} 0 8
[junit4:junit4]   2> 20658 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[26 (1430173164213633024)]} 0 1
[junit4:junit4]   2> 20658 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[26 (1430173164213633024)]} 0 1
[junit4:junit4]   2> 20659 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[26 (1430173164213633024)]} 0 5
[junit4:junit4]   2> 20660 T431 C27 P55443 /update {version=2&wt=javabin} {add=[26]} 0 8
[junit4:junit4]   2> 20668 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[27 (1430173164224118784)]} 0 2
[junit4:junit4]   2> 20668 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[27 (1430173164224118784)]} 0 2
[junit4:junit4]   2> 20669 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[27 (1430173164224118784)]} 0 5
[junit4:junit4]   2> 20669 T432 C27 P55443 /update {version=2&wt=javabin} {add=[27]} 0 7
[junit4:junit4]   2> 20678 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[28 (1430173164234604544)]} 0 2
[junit4:junit4]   2> 20678 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[28 (1430173164234604544)]} 0 1
[junit4:junit4]   2> 20678 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[28 (1430173164234604544)]} 0 5
[junit4:junit4]   2> 20679 T430 C27 P55443 /update {version=2&wt=javabin} {add=[28]} 0 7
[junit4:junit4]   2> 20687 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[29 (1430173164244041728)]} 0 1
[junit4:junit4]   2> 20687 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[29 (1430173164244041728)]} 0 1
[junit4:junit4]   2> 20687 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[29 (1430173164244041728)]} 0 4
[junit4:junit4]   2> 20688 T429 C27 P55443 /update {version=2&wt=javabin} {add=[29]} 0 7
[junit4:junit4]   2> 20695 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[30 (1430173164253478912)]} 0 1
[junit4:junit4]   2> 20695 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[30 (1430173164253478912)]} 0 1
[junit4:junit4]   2> 20696 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[30 (1430173164253478912)]} 0 5
[junit4:junit4]   2> 20696 T431 C27 P55443 /update {version=2&wt=javabin} {add=[30]} 0 6
[junit4:junit4]   2> 20704 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[31 (1430173164261867520)]} 0 1
[junit4:junit4]   2> 20704 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[31 (1430173164261867520)]} 0 1
[junit4:junit4]   2> 20705 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[31 (1430173164261867520)]} 0 5
[junit4:junit4]   2> 20705 T432 C27 P55443 /update {version=2&wt=javabin} {add=[31]} 0 7
[junit4:junit4]   2> 20714 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[32 (1430173164272353280)]} 0 1
[junit4:junit4]   2> 20714 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[32 (1430173164272353280)]} 0 1
[junit4:junit4]   2> 20714 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[32 (1430173164272353280)]} 0 5
[junit4:junit4]   2> 20715 T430 C27 P55443 /update {version=2&wt=javabin} {add=[32]} 0 8
[junit4:junit4]   2> 20723 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[33 (1430173164281790464)]} 0 1
[junit4:junit4]   2> 20723 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[33 (1430173164281790464)]} 0 1
[junit4:junit4]   2> 20724 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[33 (1430173164281790464)]} 0 5
[junit4:junit4]   2> 20725 T429 C27 P55443 /update {version=2&wt=javabin} {add=[33]} 0 8
[junit4:junit4]   2> 20733 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[34 (1430173164292276224)]} 0 1
[junit4:junit4]   2> 20733 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[34 (1430173164292276224)]} 0 1
[junit4:junit4]   2> 20734 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[34 (1430173164292276224)]} 0 5
[junit4:junit4]   2> 20734 T431 C27 P55443 /update {version=2&wt=javabin} {add=[34]} 0 7
[junit4:junit4]   2> 20743 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[35 (1430173164302761984)]} 0 1
[junit4:junit4]   2> 20743 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[35 (1430173164302761984)]} 0 1
[junit4:junit4]   2> 20744 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[35 (1430173164302761984)]} 0 5
[junit4:junit4]   2> 20744 T432 C27 P55443 /update {version=2&wt=javabin} {add=[35]} 0 7
[junit4:junit4]   2> 20752 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[36 (1430173164313247744)]} 0 1
[junit4:junit4]   2> 20752 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[36 (1430173164313247744)]} 0 1
[junit4:junit4]   2> 20753 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[36 (1430173164313247744)]} 0 4
[junit4:junit4]   2> 20754 T430 C27 P55443 /update {version=2&wt=javabin} {add=[36]} 0 7
[junit4:junit4]   2> 20762 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[37 (1430173164322684928)]} 0 1
[junit4:junit4]   2> 20762 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[37 (1430173164322684928)]} 0 1
[junit4:junit4]   2> 20763 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[37 (1430173164322684928)]} 0 5
[junit4:junit4]   2> 20763 T429 C27 P55443 /update {version=2&wt=javabin} {add=[37]} 0 7
[junit4:junit4]   2> 20771 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[38 (1430173164332122112)]} 0 1
[junit4:junit4]   2> 20771 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[38 (1430173164332122112)]} 0 1
[junit4:junit4]   2> 20771 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[38 (1430173164332122112)]} 0 4
[junit4:junit4]   2> 20772 T431 C27 P55443 /update {version=2&wt=javabin} {add=[38]} 0 7
[junit4:junit4]   2> 20780 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[39 (1430173164341559296)]} 0 1
[junit4:junit4]   2> 20780 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[39 (1430173164341559296)]} 0 1
[junit4:junit4]   2> 20781 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[39 (1430173164341559296)]} 0 5
[junit4:junit4]   2> 20781 T432 C27 P55443 /update {version=2&wt=javabin} {add=[39]} 0 7
[junit4:junit4]   2> 20788 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[40 (1430173164350996480)]} 0 0
[junit4:junit4]   2> 20789 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[40 (1430173164350996480)]} 0 1
[junit4:junit4]   2> 20790 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[40 (1430173164350996480)]} 0 5
[junit4:junit4]   2> 20790 T430 C27 P55443 /update {version=2&wt=javabin} {add=[40]} 0 7
[junit4:junit4]   2> 20798 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[41 (1430173164360433664)]} 0 2
[junit4:junit4]   2> 20798 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[41 (1430173164360433664)]} 0 2
[junit4:junit4]   2> 20799 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[41 (1430173164360433664)]} 0 5
[junit4:junit4]   2> 20799 T429 C27 P55443 /update {version=2&wt=javabin} {add=[41]} 0 7
[junit4:junit4]   2> 20807 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[42 (1430173164369870848)]} 0 1
[junit4:junit4]   2> 20807 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[42 (1430173164369870848)]} 0 1
[junit4:junit4]   2> 20808 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[42 (1430173164369870848)]} 0 5
[junit4:junit4]   2> 20808 T431 C27 P55443 /update {version=2&wt=javabin} {add=[42]} 0 7
[junit4:junit4]   2> 20816 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[43 (1430173164379308032)]} 0 1
[junit4:junit4]   2> 20816 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[43 (1430173164379308032)]} 0 1
[junit4:junit4]   2> 20817 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[43 (1430173164379308032)]} 0 5
[junit4:junit4]   2> 20818 T432 C27 P55443 /update {version=2&wt=javabin} {add=[43]} 0 6
[junit4:junit4]   2> 20825 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[44 (1430173164389793792)]} 0 1
[junit4:junit4]   2> 20826 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[44 (1430173164389793792)]} 0 2
[junit4:junit4]   2> 20826 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[44 (1430173164389793792)]} 0 5
[junit4:junit4]   2> 20827 T430 C27 P55443 /update {version=2&wt=javabin} {add=[44]} 0 7
[junit4:junit4]   2> 20835 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[45 (1430173164399230976)]} 0 1
[junit4:junit4]   2> 20835 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[45 (1430173164399230976)]} 0 1
[junit4:junit4]   2> 20836 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[45 (1430173164399230976)]} 0 5
[junit4:junit4]   2> 20836 T429 C27 P55443 /update {version=2&wt=javabin} {add=[45]} 0 7
[junit4:junit4]   2> 20848 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[46 (1430173164409716736)]} 0 1
[junit4:junit4]   2> 20848 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[46 (1430173164409716736)]} 0 1
[junit4:junit4]   2> 20849 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[46 (1430173164409716736)]} 0 8
[junit4:junit4]   2> 20849 T431 C27 P55443 /update {version=2&wt=javabin} {add=[46]} 0 10
[junit4:junit4]   2> 20858 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[47 (1430173164423348224)]} 0 1
[junit4:junit4]   2> 20858 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[47 (1430173164423348224)]} 0 1
[junit4:junit4]   2> 20859 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[47 (1430173164423348224)]} 0 5
[junit4:junit4]   2> 20859 T432 C27 P55443 /update {version=2&wt=javabin} {add=[47]} 0 7
[junit4:junit4]   2> 20867 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[48 (1430173164432785408)]} 0 1
[junit4:junit4]   2> 20867 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[48 (1430173164432785408)]} 0 1
[junit4:junit4]   2> 20868 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[48 (1430173164432785408)]} 0 5
[junit4:junit4]   2> 20869 T430 C27 P55443 /update {version=2&wt=javabin} {add=[48]} 0 8
[junit4:junit4]   2> 20877 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[49 (1430173164443271168)]} 0 1
[junit4:junit4]   2> 20877 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[49 (1430173164443271168)]} 0 1
[junit4:junit4]   2> 20877 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[49 (1430173164443271168)]} 0 4
[junit4:junit4]   2> 20878 T429 C27 P55443 /update {version=2&wt=javabin} {add=[49]} 0 7
[junit4:junit4]   2> 20885 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[50 (1430173164452708352)]} 0 1
[junit4:junit4]   2> 20885 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[50 (1430173164452708352)]} 0 1
[junit4:junit4]   2> 20886 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[50 (1430173164452708352)]} 0 4
[junit4:junit4]   2> 20886 T431 C27 P55443 /update {version=2&wt=javabin} {add=[50]} 0 6
[junit4:junit4]   2> 20894 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[51 (1430173164461096960)]} 0 1
[junit4:junit4]   2> 20894 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[51 (1430173164461096960)]} 0 1
[junit4:junit4]   2> 20895 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[51 (1430173164461096960)]} 0 5
[junit4:junit4]   2> 20896 T432 C27 P55443 /update {version=2&wt=javabin} {add=[51]} 0 8
[junit4:junit4]   2> 20903 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[52 (1430173164471582720)]} 0 0
[junit4:junit4]   2> 20903 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[52 (1430173164471582720)]} 0 0
[junit4:junit4]   2> 20904 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[52 (1430173164471582720)]} 0 4
[junit4:junit4]   2> 20905 T430 C27 P55443 /update {version=2&wt=javabin} {add=[52]} 0 7
[junit4:junit4]   2> 20912 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[53 (1430173164481019904)]} 0 1
[junit4:junit4]   2> 20913 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[53 (1430173164481019904)]} 0 1
[junit4:junit4]   2> 20913 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[53 (1430173164481019904)]} 0 4
[junit4:junit4]   2> 20914 T429 C27 P55443 /update {version=2&wt=javabin} {add=[53]} 0 7
[junit4:junit4]   2> 20921 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[54 (1430173164490457088)]} 0 1
[junit4:junit4]   2> 20921 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[54 (1430173164490457088)]} 0 1
[junit4:junit4]   2> 20922 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[54 (1430173164490457088)]} 0 4
[junit4:junit4]   2> 20922 T431 C27 P55443 /update {version=2&wt=javabin} {add=[54]} 0 6
[junit4:junit4]   2> 20930 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[55 (1430173164499894272)]} 0 1
[junit4:junit4]   2> 20930 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[55 (1430173164499894272)]} 0 1
[junit4:junit4]   2> 20931 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[55 (1430173164499894272)]} 0 5
[junit4:junit4]   2> 20931 T432 C27 P55443 /update {version=2&wt=javabin} {add=[55]} 0 7
[junit4:junit4]   2> 20939 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[56 (1430173164509331456)]} 0 1
[junit4:junit4]   2> 20939 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[56 (1430173164509331456)]} 0 1
[junit4:junit4]   2> 20940 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[56 (1430173164509331456)]} 0 5
[junit4:junit4]   2> 20941 T430 C27 P55443 /update {version=2&wt=javabin} {add=[56]} 0 7
[junit4:junit4]   2> 20949 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[57 (1430173164518768640)]} 0 1
[junit4:junit4]   2> 20949 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[57 (1430173164518768640)]} 0 1
[junit4:junit4]   2> 20950 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[57 (1430173164518768640)]} 0 5
[junit4:junit4]   2> 20950 T432 C27 P55443 /update {version=2&wt=javabin} {add=[57]} 0 7
[junit4:junit4]   2> 20958 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[58 (1430173164529254400)]} 0 1
[junit4:junit4]   2> 20958 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[58 (1430173164529254400)]} 0 1
[junit4:junit4]   2> 20959 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[58 (1430173164529254400)]} 0 5
[junit4:junit4]   2> 20959 T430 C27 P55443 /update {version=2&wt=javabin} {add=[58]} 0 6
[junit4:junit4]   2> 20967 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[59 (1430173164537643008)]} 0 1
[junit4:junit4]   2> 20967 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[59 (1430173164537643008)]} 0 1
[junit4:junit4]   2> 20967 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[59 (1430173164537643008)]} 0 4
[junit4:junit4]   2> 20968 T431 C27 P55443 /update {version=2&wt=javabin} {add=[59]} 0 7
[junit4:junit4]   2> 20985 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[60 (1430173164555468800)]} 0 1
[junit4:junit4]   2> 20986 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[60 (1430173164555468800)]} 0 1
[junit4:junit4]   2> 20987 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[60 (1430173164555468800)]} 0 7
[junit4:junit4]   2> 20988 T429 C27 P55443 /update {version=2&wt=javabin} {add=[60]} 0 10
[junit4:junit4]   2> 20995 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[61 (1430173164568051712)]} 0 1
[junit4:junit4]   2> 20995 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[61 (1430173164568051712)]} 0 1
[junit4:junit4]   2> 20996 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[61 (1430173164568051712)]} 0 4
[junit4:junit4]   2> 20996 T432 C27 P55443 /update {version=2&wt=javabin} {add=[61]} 0 6
[junit4:junit4]   2> 21004 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[62 (1430173164577488896)]} 0 1
[junit4:junit4]   2> 21004 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[62 (1430173164577488896)]} 0 1
[junit4:junit4]   2> 21005 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[62 (1430173164577488896)]} 0 5
[junit4:junit4]   2> 21005 T430 C27 P55443 /update {version=2&wt=javabin} {add=[62]} 0 6
[junit4:junit4]   2> 21013 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[63 (1430173164585877504)]} 0 1
[junit4:junit4]   2> 21013 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[63 (1430173164585877504)]} 0 1
[junit4:junit4]   2> 21014 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[63 (1430173164585877504)]} 0 5
[junit4:junit4]   2> 21014 T431 C27 P55443 /update {version=2&wt=javabin} {add=[63]} 0 7
[junit4:junit4]   2> 21021 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[64 (1430173164595314688)]} 0 1
[junit4:junit4]   2> 21021 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[64 (1430173164595314688)]} 0 1
[junit4:junit4]   2> 21022 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[64 (1430173164595314688)]} 0 4
[junit4:junit4]   2> 21023 T429 C27 P55443 /update {version=2&wt=javabin} {add=[64]} 0 7
[junit4:junit4]   2> 21031 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[65 (1430173164604751872)]} 0 1
[junit4:junit4]   2> 21031 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[65 (1430173164604751872)]} 0 1
[junit4:junit4]   2> 21032 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[65 (1430173164604751872)]} 0 5
[junit4:junit4]   2> 21032 T432 C27 P55443 /update {version=2&wt=javabin} {add=[65]} 0 7
[junit4:junit4]   2> 21040 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[66 (1430173164614189056)]} 0 1
[junit4:junit4]   2> 21040 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[66 (1430173164614189056)]} 0 1
[junit4:junit4]   2> 21041 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[66 (1430173164614189056)]} 0 5
[junit4:junit4]   2> 21041 T430 C27 P55443 /update {version=2&wt=javabin} {add=[66]} 0 7
[junit4:junit4]   2> 21048 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[67 (1430173164623626240)]} 0 1
[junit4:junit4]   2> 21048 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[67 (1430173164623626240)]} 0 0
[junit4:junit4]   2> 21049 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[67 (1430173164623626240)]} 0 4
[junit4:junit4]   2> 21049 T431 C27 P55443 /update {version=2&wt=javabin} {add=[67]} 0 6
[junit4:junit4]   2> 21057 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[68 (1430173164633063424)]} 0 1
[junit4:junit4]   2> 21057 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[68 (1430173164633063424)]} 0 1
[junit4:junit4]   2> 21058 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[68 (1430173164633063424)]} 0 5
[junit4:junit4]   2> 21058 T429 C27 P55443 /update {version=2&wt=javabin} {add=[68]} 0 7
[junit4:junit4]   2> 21065 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[69 (1430173164641452032)]} 0 0
[junit4:junit4]   2> 21065 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[69 (1430173164641452032)]} 0 0
[junit4:junit4]   2> 21066 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[69 (1430173164641452032)]} 0 4
[junit4:junit4]   2> 21067 T432 C27 P55443 /update {version=2&wt=javabin} {add=[69]} 0 7
[junit4:junit4]   2> 21074 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[70 (1430173164650889216)]} 0 1
[junit4:junit4]   2> 21074 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[70 (1430173164650889216)]} 0 0
[junit4:junit4]   2> 21075 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[70 (1430173164650889216)]} 0 4
[junit4:junit4]   2> 21076 T430 C27 P55443 /update {version=2&wt=javabin} {add=[70]} 0 7
[junit4:junit4]   2> 21083 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[71 (1430173164660326400)]} 0 1
[junit4:junit4]   2> 21083 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[71 (1430173164660326400)]} 0 1
[junit4:junit4]   2> 21083 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[71 (1430173164660326400)]} 0 4
[junit4:junit4]   2> 21084 T431 C27 P55443 /update {version=2&wt=javabin} {add=[71]} 0 6
[junit4:junit4]   2> 21091 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[72 (1430173164668715008)]} 0 1
[junit4:junit4]   2> 21091 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[72 (1430173164668715008)]} 0 1
[junit4:junit4]   2> 21092 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[72 (1430173164668715008)]} 0 4
[junit4:junit4]   2> 21092 T429 C27 P55443 /update {version=2&wt=javabin} {add=[72]} 0 6
[junit4:junit4]   2> 21099 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[73 (1430173164677103616)]} 0 1
[junit4:junit4]   2> 21099 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[73 (1430173164677103616)]} 0 1
[junit4:junit4]   2> 21099 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[73 (1430173164677103616)]} 0 3
[junit4:junit4]   2> 21100 T432 C27 P55443 /update {version=2&wt=javabin} {add=[73]} 0 6
[junit4:junit4]   2> 21107 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[74 (1430173164685492224)]} 0 1
[junit4:junit4]   2> 21107 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[74 (1430173164685492224)]} 0 1
[junit4:junit4]   2> 21108 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[74 (1430173164685492224)]} 0 4
[junit4:junit4]   2> 21108 T430 C27 P55443 /update {version=2&wt=javabin} {add=[74]} 0 6
[junit4:junit4]   2> 21116 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[75 (1430173164693880832)]} 0 1
[junit4:junit4]   2> 21117 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[75 (1430173164693880832)]} 0 1
[junit4:junit4]   2> 21117 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[75 (1430173164693880832)]} 0 5
[junit4:junit4]   2> 21118 T431 C27 P55443 /update {version=2&wt=javabin} {add=[75]} 0 8
[junit4:junit4]   2> 21126 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[76 (1430173164704366592)]} 0 1
[junit4:junit4]   2> 21126 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[76 (1430173164704366592)]} 0 1
[junit4:junit4]   2> 21127 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[76 (1430173164704366592)]} 0 5
[junit4:junit4]   2> 21127 T429 C27 P55443 /update {version=2&wt=javabin} {add=[76]} 0 7
[junit4:junit4]   2> 21135 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[77 (1430173164713803776)]} 0 1
[junit4:junit4]   2> 21135 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[77 (1430173164713803776)]} 0 1
[junit4:junit4]   2> 21136 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[77 (1430173164713803776)]} 0 5
[junit4:junit4]   2> 21136 T432 C27 P55443 /update {version=2&wt=javabin} {add=[77]} 0 7
[junit4:junit4]   2> 21143 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[78 (1430173164724289536)]} 0 0
[junit4:junit4]   2> 21144 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[78 (1430173164724289536)]} 0 1
[junit4:junit4]   2> 21144 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[78 (1430173164724289536)]} 0 4
[junit4:junit4]   2> 21145 T430 C27 P55443 /update {version=2&wt=javabin} {add=[78]} 0 6
[junit4:junit4]   2> 21152 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[79 (1430173164732678144)]} 0 1
[junit4:junit4]   2> 21152 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[79 (1430173164732678144)]} 0 1
[junit4:junit4]   2> 21153 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[79 (1430173164732678144)]} 0 5
[junit4:junit4]   2> 21153 T431 C27 P55443 /update {version=2&wt=javabin} {add=[79]} 0 6
[junit4:junit4]   2> 21160 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[80 (1430173164741066752)]} 0 1
[junit4:junit4]   2> 21160 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[80 (1430173164741066752)]} 0 1
[junit4:junit4]   2> 21161 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[80 (1430173164741066752)]} 0 4
[junit4:junit4]   2> 21161 T429 C27 P55443 /update {version=2&wt=javabin} {add=[80]} 0 6
[junit4:junit4]   2> 21168 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[81 (1430173164749455360)]} 0 1
[junit4:junit4]   2> 21168 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[81 (1430173164749455360)]} 0 0
[junit4:junit4]   2> 21169 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[81 (1430173164749455360)]} 0 4
[junit4:junit4]   2> 21170 T432 C27 P55443 /update {version=2&wt=javabin} {add=[81]} 0 7
[junit4:junit4]   2> 21176 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[82 (1430173164757843968)]} 0 0
[junit4:junit4]   2> 21176 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[82 (1430173164757843968)]} 0 0
[junit4:junit4]   2> 21177 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[82 (1430173164757843968)]} 0 4
[junit4:junit4]   2> 21177 T430 C27 P55443 /update {version=2&wt=javabin} {add=[82]} 0 5
[junit4:junit4]   2> 21184 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[83 (1430173164766232576)]} 0 0
[junit4:junit4]   2> 21185 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[83 (1430173164766232576)]} 0 1
[junit4:junit4]   2> 21185 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[83 (1430173164766232576)]} 0 4
[junit4:junit4]   2> 21186 T431 C27 P55443 /update {version=2&wt=javabin} {add=[83]} 0 7
[junit4:junit4]   2> 21192 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[84 (1430173164775669760)]} 0 0
[junit4:junit4]   2> 21192 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[84 (1430173164775669760)]} 0 0
[junit4:junit4]   2> 21193 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[84 (1430173164775669760)]} 0 4
[junit4:junit4]   2> 21193 T429 C27 P55443 /update {version=2&wt=javabin} {add=[84]} 0 5
[junit4:junit4]   2> 21200 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[85 (1430173164783009792)]} 0 1
[junit4:junit4]   2> 21200 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[85 (1430173164783009792)]} 0 1
[junit4:junit4]   2> 21201 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[85 (1430173164783009792)]} 0 4
[junit4:junit4]   2> 21201 T432 C27 P55443 /update {version=2&wt=javabin} {add=[85]} 0 6
[junit4:junit4]   2> 21208 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[86 (1430173164791398400)]} 0 1
[junit4:junit4]   2> 21208 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[86 (1430173164791398400)]} 0 1
[junit4:junit4]   2> 21209 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[86 (1430173164791398400)]} 0 4
[junit4:junit4]   2> 21209 T430 C27 P55443 /update {version=2&wt=javabin} {add=[86]} 0 6
[junit4:junit4]   2> 21216 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[87 (1430173164799787008)]} 0 1
[junit4:junit4]   2> 21216 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[87 (1430173164799787008)]} 0 1
[junit4:junit4]   2> 21217 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[87 (1430173164799787008)]} 0 4
[junit4:junit4]   2> 21217 T431 C27 P55443 /update {version=2&wt=javabin} {add=[87]} 0 6
[junit4:junit4]   2> 21224 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[88 (1430173164808175616)]} 0 1
[junit4:junit4]   2> 21224 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[88 (1430173164808175616)]} 0 1
[junit4:junit4]   2> 21224 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[88 (1430173164808175616)]} 0 3
[junit4:junit4]   2> 21225 T429 C27 P55443 /update {version=2&wt=javabin} {add=[88]} 0 6
[junit4:junit4]   2> 21232 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[89 (1430173164816564224)]} 0 1
[junit4:junit4]   2> 21232 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[89 (1430173164816564224)]} 0 1
[junit4:junit4]   2> 21233 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[89 (1430173164816564224)]} 0 4
[junit4:junit4]   2> 21233 T432 C27 P55443 /update {version=2&wt=javabin} {add=[89]} 0 6
[junit4:junit4]   2> 21240 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[90 (1430173164826001408)]} 0 0
[junit4:junit4]   2> 21241 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[90 (1430173164826001408)]} 0 1
[junit4:junit4]   2> 21241 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[90 (1430173164826001408)]} 0 4
[junit4:junit4]   2> 21242 T430 C27 P55443 /update {version=2&wt=javabin} {add=[90]} 0 6
[junit4:junit4]   2> 21249 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[91 (1430173164834390016)]} 0 0
[junit4:junit4]   2> 21249 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[91 (1430173164834390016)]} 0 1
[junit4:junit4]   2> 21249 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[91 (1430173164834390016)]} 0 4
[junit4:junit4]   2> 21250 T431 C27 P55443 /update {version=2&wt=javabin} {add=[91]} 0 6
[junit4:junit4]   2> 21256 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[92 (1430173164842778624)]} 0 0
[junit4:junit4]   2> 21256 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[92 (1430173164842778624)]} 0 0
[junit4:junit4]   2> 21257 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[92 (1430173164842778624)]} 0 3
[junit4:junit4]   2> 21258 T429 C27 P55443 /update {version=2&wt=javabin} {add=[92]} 0 6
[junit4:junit4]   2> 21265 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[93 (1430173164851167232)]} 0 1
[junit4:junit4]   2> 21268 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[93 (1430173164851167232)]} 0 4
[junit4:junit4]   2> 21268 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[93 (1430173164851167232)]} 0 7
[junit4:junit4]   2> 21269 T432 C27 P55443 /update {version=2&wt=javabin} {add=[93]} 0 9
[junit4:junit4]   2> 21276 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[94 (1430173164862701568)]} 0 1
[junit4:junit4]   2> 21276 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[94 (1430173164862701568)]} 0 1
[junit4:junit4]   2> 21277 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[94 (1430173164862701568)]} 0 4
[junit4:junit4]   2>

[...truncated too long message...]

afe stop:
[junit4:junit4]   1> safe stop:
[junit4:junit4]   1> safe stop:
[junit4:junit4]   2> 34488 T747 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 34491 T746 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 34588 T750 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4]   2> 35703 T831 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 35703 T831 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 35703 T831 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 35703 T831 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ga, timezone=America/Belem
[junit4:junit4]   2> NOTE: Linux 3.2.0-39-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=1,free=94319912,total=282460160
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestSweetSpotSimilarityFactory, SynonymTokenizerTest, SearchHandlerTest, ZkControllerTest, TestDocSet, DistributedQueryElevationComponentTest, AlternateDirectoryTest, FileBasedSpellCheckerTest, TestPluginEnable, SuggesterTest, QueryEqualityTest, DistributedTermsComponentTest, TestLFUCache, MoreLikeThisHandlerTest, TestCoreContainer, TestCopyFieldCollectionResource, TestFieldTypeResource, TestLRUCache, TestSolrCoreProperties, TestBadConfig, BasicFunctionalityTest, RAMDirectoryFactoryTest, FieldMutatingUpdateProcessorTest, TestXIncludeConfig, TestQueryUtils, LoggingHandlerTest, UnloadDistributedZkTest, TermVectorComponentTest, QueryParsingTest, TestCharFilters, ResourceLoaderTest, AutoCommitTest, TestRandomDVFaceting, CSVRequestHandlerTest, RegexBoostProcessorTest, TestIBSimilarityFactory, PreAnalyzedFieldTest, CoreAdminHandlerTest, DocValuesTest, OverseerCollectionProcessorTest, RecoveryZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RecoveryZkTest -Dtests.seed=8A99804DB37337B1 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ga -Dtests.timezone=America/Belem -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J0 | RecoveryZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=806, name=coreLoadExecutor-457-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
[junit4:junit4]    > 	at java.lang.Thread.getStackTrace(Thread.java:1569)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:524)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:991)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:883)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:875)
[junit4:junit4]    > 	at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:37)
[junit4:junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:327)
[junit4:junit4]    > 	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:244)
[junit4:junit4]    > 	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:458)
[junit4:junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:746)
[junit4:junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:619)
[junit4:junit4]    > 	at org.apache.solr.core.CoreContainer.createFromZk(CoreContainer.java:872)
[junit4:junit4]    > 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:955)
[junit4:junit4]    > 	at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:579)
[junit4:junit4]    > 	at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:574)
[junit4:junit4]    > 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4:junit4]    > 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4:junit4]    > 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4:junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
[junit4:junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.RAMDirectory@3e2c64 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1588dba)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	... 1 more
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=808, name=RecoveryThread, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
[junit4:junit4]    > 	at java.lang.Thread.getStackTrace(Thread.java:1569)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:524)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:991)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:883)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:875)
[junit4:junit4]    > 	at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:37)
[junit4:junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:327)
[junit4:junit4]    > 	at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:383)
[junit4:junit4]    > 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:281)
[junit4:junit4]    > 	at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:153)
[junit4:junit4]    > 	at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409)
[junit4:junit4]    > 	at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:223)
[junit4:junit4]    > Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ca4702 lockFactory=org.apache.lucene.store.NativeFSLockFactory@41fbf7)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] Completed on J0 in 35.79s, 1 test, 2 errors <<< FAILURES!

[...truncated 519 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:375: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:355: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1210: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:874: There were test failures: 272 suites, 1145 tests, 2 suite-level errors, 1 failure, 13 ignored (7 assumptions)

Total time: 37 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b79 -client -XX:+UseParallelGC
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message