Return-Path: X-Original-To: apmail-lucene-dev-archive@www.apache.org Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 787BCF2B6 for ; Fri, 22 Mar 2013 02:50:06 +0000 (UTC) Received: (qmail 50610 invoked by uid 500); 22 Mar 2013 02:50:04 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 50397 invoked by uid 500); 22 Mar 2013 02:50:04 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 50390 invoked by uid 99); 22 Mar 2013 02:50:04 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Mar 2013 02:50:04 +0000 X-ASF-Spam-Status: No, hits=0.0 required=5.0 tests=NORMAL_HTTP_TO_IP,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jenkins@thetaphi.de designates 188.138.97.18 as permitted sender) Received: from [188.138.97.18] (HELO mail.sd-datasolutions.de) (188.138.97.18) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Mar 2013 02:49:54 +0000 Received: from serv1 (localhost.localdomain [127.0.0.1]) by mail.sd-datasolutions.de (Postfix) with ESMTP id 0D05114AA009; Fri, 22 Mar 2013 02:49:32 +0000 (UTC) Date: Fri, 22 Mar 2013 02:49:16 +0000 (UTC) From: Policeman Jenkins Server To: dev@lucene.apache.org, rmuir@apache.org Message-ID: <1711757749.101.1363920572018.JavaMail.jenkins@serv1.sd-datasolutions.de> Subject: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b79) - Build # 4817 - Failure! MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_100_306175094.1363920556112" X-Jenkins-Job: Lucene-Solr-trunk-Linux X-Jenkins-Result: FAILURE X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_100_306175094.1363920556112 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/4817/ Java: 32bit/jdk1.8.0-ea-b79 -client -XX:+UseParallelGC 3 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest Error Message: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=806, name=coreLoadExecutor-457-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below. Stack Trace: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=806, name=coreLoadExecutor-457-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below. at java.lang.Thread.getStackTrace(Thread.java:1569) at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150) at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:524) at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:991) at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:883) at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:875) at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:37) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:327) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:244) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:458) at org.apache.solr.core.SolrCore.(SolrCore.java:746) at org.apache.solr.core.SolrCore.(SolrCore.java:619) at org.apache.solr.core.CoreContainer.createFromZk(CoreContainer.java:872) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:955) at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:579) at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:574) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.RAMDirectory@3e2c64 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1588dba) at org.junit.Assert.fail(Assert.java:93) at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47) at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602) at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599) at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) ... 1 more FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest Error Message: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=808, name=RecoveryThread, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below. Stack Trace: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=808, name=RecoveryThread, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below. at java.lang.Thread.getStackTrace(Thread.java:1569) at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150) at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:524) at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:991) at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:883) at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:875) at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:37) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:327) at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:383) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:281) at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:153) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:223) Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ca4702 lockFactory=org.apache.lucene.store.NativeFSLockFactory@41fbf7) at org.junit.Assert.fail(Assert.java:93) at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47) at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602) at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599) at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at java.lang.Thread.run(Thread.java:722) REGRESSION: org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch Error Message: Still found shard2 in collection test_unload_shard_and_collection Stack Trace: java.lang.AssertionError: Still found shard2 in collection test_unload_shard_and_collection at __randomizedtesting.SeedInfo.seed([8A99804DB37337B1:B7F0E55C42C578D]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:124) at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:487) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at java.lang.Thread.run(Thread.java:722) Build Log: [...truncated 9099 lines...] [junit4:junit4] Suite: org.apache.solr.cloud.UnloadDistributedZkTest [junit4:junit4] 2> 0 T368 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /iw_/ [junit4:junit4] 2> 5 T368 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076 [junit4:junit4] 2> 6 T368 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4:junit4] 2> 7 T369 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4:junit4] 2> 10 T369 oazs.ZooKeeperServer.setTickTime tickTime set to 1000 [junit4:junit4] 2> 10 T369 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1 [junit4:junit4] 2> 10 T369 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1 [junit4:junit4] 2> 11 T369 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0 [junit4:junit4] 2> 12 T369 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-1363919394076/zookeeper/server1/data/version-2/snapshot.0 [junit4:junit4] 2> 107 T368 oasc.ZkTestServer.run start zk server on port:53441 [junit4:junit4] 2> 108 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@824ddb [junit4:junit4] 2> 109 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 113 T374 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 114 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36563 [junit4:junit4] 2> 115 T374 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 115 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36563 [junit4:junit4] 2> 116 T372 oazsp.FileTxnLog.append Creating new log file: log.1 [junit4:junit4] 2> 228 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230000 with negotiated timeout 10000 for client /127.0.0.1:36563 [junit4:junit4] 2> 228 T374 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230000, negotiated timeout = 10000 [junit4:junit4] 2> 228 T375 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@824ddb name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 229 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 229 T368 oascc.SolrZkClient.makePath makePath: /solr [junit4:junit4] 2> 232 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230000 [junit4:junit4] 2> 233 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36563 which had sessionid 0x13d8fee55230000 [junit4:junit4] 2> 233 T375 oaz.ClientCnxn$EventThread.run EventThread shut down [junit4:junit4] 2> 233 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230000 closed [junit4:junit4] 2> 234 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@45b73b [junit4:junit4] 2> 235 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 235 T376 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 235 T376 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 235 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36564 [junit4:junit4] 2> 236 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36564 [junit4:junit4] 2> 237 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230001 with negotiated timeout 10000 for client /127.0.0.1:36564 [junit4:junit4] 2> 237 T376 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230001, negotiated timeout = 10000 [junit4:junit4] 2> 237 T377 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45b73b name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 237 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 238 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4:junit4] 2> 241 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4:junit4] 2> 244 T368 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4:junit4] 2> 246 T368 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4:junit4] 2> 249 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4:junit4] 2> 250 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4:junit4] 2> 255 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4:junit4] 2> 256 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4:junit4] 2> 263 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4:junit4] 2> 264 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4:junit4] 2> 269 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4:junit4] 2> 269 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4:junit4] 2> 273 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4:junit4] 2> 274 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4:junit4] 2> 277 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4:junit4] 2> 278 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4:junit4] 2> 280 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4:junit4] 2> 281 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4:junit4] 2> 284 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4:junit4] 2> 284 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4:junit4] 2> 287 T368 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4:junit4] 2> 287 T368 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4:junit4] 2> 290 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230001 [junit4:junit4] 2> 290 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36564 which had sessionid 0x13d8fee55230001 [junit4:junit4] 2> 290 T377 oaz.ClientCnxn$EventThread.run EventThread shut down [junit4:junit4] 2> 290 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230001 closed [junit4:junit4] 2> 369 T368 oejs.Server.doStart jetty-8.1.8.v20121106 [junit4:junit4] 2> 379 T368 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47038 [junit4:junit4] 2> 380 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4:junit4] 2> 380 T368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4:junit4] 2> 381 T368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1363919394361 [junit4:junit4] 2> 381 T368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1363919394361/solr.xml [junit4:junit4] 2> 382 T368 oasc.CoreContainer. New CoreContainer 16251349 [junit4:junit4] 2> 382 T368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1363919394361/' [junit4:junit4] 2> 382 T368 oasc.SolrResourceLoader. new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1363919394361/' [junit4:junit4] 2> 406 T368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000 [junit4:junit4] 2> 407 T368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http:// [junit4:junit4] 2> 407 T368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4:junit4] 2> 408 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4:junit4] 2> 408 T368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4:junit4] 2> 409 T368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4:junit4] 2> 409 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4:junit4] 2> 410 T368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4:junit4] 2> 410 T368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4:junit4] 2> 411 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false [junit4:junit4] 2> 423 T368 oasc.CoreContainer.load Registering Log Listener [junit4:junit4] 2> 444 T368 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53441/solr [junit4:junit4] 2> 444 T368 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4:junit4] 2> 445 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@cf64e1 [junit4:junit4] 2> 446 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 446 T387 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 447 T387 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 447 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36565 [junit4:junit4] 2> 447 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36565 [junit4:junit4] 2> 448 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230002 with negotiated timeout 20000 for client /127.0.0.1:36565 [junit4:junit4] 2> 448 T387 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230002, negotiated timeout = 20000 [junit4:junit4] 2> 448 T388 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cf64e1 name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 449 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 450 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230002 [junit4:junit4] 2> 450 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36565 which had sessionid 0x13d8fee55230002 [junit4:junit4] 2> 450 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230002 closed [junit4:junit4] 2> 451 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000 [junit4:junit4] 2> 452 T388 oaz.ClientCnxn$EventThread.run EventThread shut down [junit4:junit4] 2> 455 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7d1b06 [junit4:junit4] 2> 456 T389 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 456 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 457 T389 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 457 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36566 [junit4:junit4] 2> 458 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36566 [junit4:junit4] 2> 458 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230003 with negotiated timeout 20000 for client /127.0.0.1:36566 [junit4:junit4] 2> 458 T389 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230003, negotiated timeout = 20000 [junit4:junit4] 2> 459 T390 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d1b06 name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 459 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 461 T368 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4:junit4] 2> 464 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:create cxid:0x7 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 465 T368 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4:junit4] 2> 469 T368 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4:junit4] 2> 470 T368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47038_iw_ [junit4:junit4] 2> 471 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:delete cxid:0x12 zxid:0x1e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:47038_iw_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:47038_iw_ [junit4:junit4] 2> 472 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47038_iw_ [junit4:junit4] 2> 478 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4:junit4] 2> 489 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:delete cxid:0x20 zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader [junit4:junit4] 2> 490 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4:junit4] 2> 493 T368 oasc.Overseer.start Overseer (id=89385821410623491-127.0.0.1:47038_iw_-n_0000000000) starting [junit4:junit4] 2> 494 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:create cxid:0x25 zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 495 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:create cxid:0x26 zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 497 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:create cxid:0x28 zxid:0x27 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 499 T368 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4:junit4] 2> 504 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230003 type:create cxid:0x2e zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 506 T392 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4:junit4] 2> 506 T368 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4:junit4] 2> 511 T368 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4:junit4] 2> 514 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4:junit4] 2> 517 T391 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4:junit4] 2> 520 T368 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0 [junit4:junit4] 2> 520 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4:junit4] 2> 521 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 525 T368 oascsi.HttpClientUtil.createClient Creating new http client, config: [junit4:junit4] 2> 527 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@77d658 [junit4:junit4] 2> 528 T394 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 528 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 529 T394 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 529 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36567 [junit4:junit4] 2> 529 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36567 [junit4:junit4] 2> 530 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230004 with negotiated timeout 10000 for client /127.0.0.1:36567 [junit4:junit4] 2> 530 T394 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230004, negotiated timeout = 10000 [junit4:junit4] 2> 531 T395 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77d658 name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 531 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 532 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4:junit4] 2> 536 T368 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true [junit4:junit4] 2> 626 T368 oejs.Server.doStart jetty-8.1.8.v20121106 [junit4:junit4] 2> 630 T368 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47486 [junit4:junit4] 2> 630 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4:junit4] 2> 631 T368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4:junit4] 2> 631 T368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607 [junit4:junit4] 2> 631 T368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/solr.xml [junit4:junit4] 2> 632 T368 oasc.CoreContainer. New CoreContainer 20573689 [junit4:junit4] 2> 632 T368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/' [junit4:junit4] 2> 633 T368 oasc.SolrResourceLoader. new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/' [junit4:junit4] 2> 657 T368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000 [junit4:junit4] 2> 658 T368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http:// [junit4:junit4] 2> 658 T368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4:junit4] 2> 659 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4:junit4] 2> 659 T368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4:junit4] 2> 659 T368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4:junit4] 2> 660 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4:junit4] 2> 660 T368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4:junit4] 2> 661 T368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4:junit4] 2> 661 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false [junit4:junit4] 2> 670 T368 oasc.CoreContainer.load Registering Log Listener [junit4:junit4] 2> 686 T368 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53441/solr [junit4:junit4] 2> 687 T368 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4:junit4] 2> 687 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@cf4eaf [junit4:junit4] 2> 688 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 688 T405 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 689 T405 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 689 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36568 [junit4:junit4] 2> 690 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36568 [junit4:junit4] 2> 690 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230005 with negotiated timeout 20000 for client /127.0.0.1:36568 [junit4:junit4] 2> 690 T405 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230005, negotiated timeout = 20000 [junit4:junit4] 2> 691 T406 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cf4eaf name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 691 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 692 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230005 [junit4:junit4] 2> 693 T406 oaz.ClientCnxn$EventThread.run EventThread shut down [junit4:junit4] 2> 693 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36568 which had sessionid 0x13d8fee55230005 [junit4:junit4] 2> 693 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230005 closed [junit4:junit4] 2> 693 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000 [junit4:junit4] 2> 696 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@11e371e [junit4:junit4] 2> 698 T407 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 698 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 698 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36569 [junit4:junit4] 2> 698 T407 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 699 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36569 [junit4:junit4] 2> 699 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230006 with negotiated timeout 20000 for client /127.0.0.1:36569 [junit4:junit4] 2> 699 T407 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230006, negotiated timeout = 20000 [junit4:junit4] 2> 700 T408 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11e371e name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 700 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 701 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:create cxid:0x1 zxid:0x30 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 702 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:create cxid:0x3 zxid:0x31 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 704 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4:junit4] 2> 1707 T368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47486_iw_ [junit4:junit4] 2> 1708 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:delete cxid:0xd zxid:0x32 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:47486_iw_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:47486_iw_ [junit4:junit4] 2> 1709 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47486_iw_ [junit4:junit4] 2> 1712 T390 oascc.ZkStateReader$3.process Updating live nodes... (2) [junit4:junit4] 2> 1712 T395 oascc.ZkStateReader$3.process Updating live nodes... (2) [junit4:junit4] 2> 1712 T408 oascc.ZkStateReader$3.process Updating live nodes... (2) [junit4:junit4] 2> 1718 T368 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0 [junit4:junit4] 2> 1719 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4:junit4] 2> 1719 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 1816 T368 oejs.Server.doStart jetty-8.1.8.v20121106 [junit4:junit4] 2> 1818 T368 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51183 [junit4:junit4] 2> 1819 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4:junit4] 2> 1819 T368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4:junit4] 2> 1820 T368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792 [junit4:junit4] 2> 1820 T368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/solr.xml [junit4:junit4] 2> 1821 T368 oasc.CoreContainer. New CoreContainer 29144622 [junit4:junit4] 2> 1822 T368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/' [junit4:junit4] 2> 1822 T368 oasc.SolrResourceLoader. new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/' [junit4:junit4] 2> 1852 T368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000 [junit4:junit4] 2> 1853 T368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http:// [junit4:junit4] 2> 1853 T368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4:junit4] 2> 1854 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4:junit4] 2> 1854 T368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4:junit4] 2> 1855 T368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4:junit4] 2> 1855 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4:junit4] 2> 1856 T368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4:junit4] 2> 1856 T368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4:junit4] 2> 1857 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false [junit4:junit4] 2> 1868 T368 oasc.CoreContainer.load Registering Log Listener [junit4:junit4] 2> 1889 T368 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53441/solr [junit4:junit4] 2> 1890 T368 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4:junit4] 2> 1890 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1403692 [junit4:junit4] 2> 1892 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 1892 T419 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 1894 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36571 [junit4:junit4] 2> 1894 T419 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 1894 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36571 [junit4:junit4] 2> 1895 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230007 with negotiated timeout 20000 for client /127.0.0.1:36571 [junit4:junit4] 2> 1895 T419 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230007, negotiated timeout = 20000 [junit4:junit4] 2> 1896 T420 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1403692 name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 1896 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 1898 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230007 [junit4:junit4] 2> 1899 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36571 which had sessionid 0x13d8fee55230007 [junit4:junit4] 2> 1900 T420 oaz.ClientCnxn$EventThread.run EventThread shut down [junit4:junit4] 2> 1900 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230007 closed [junit4:junit4] 2> 1900 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000 [junit4:junit4] 2> 1905 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@85d6f2 [junit4:junit4] 2> 1907 T421 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 1907 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 1907 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36572 [junit4:junit4] 2> 1907 T421 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 1908 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36572 [junit4:junit4] 2> 1909 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230008 with negotiated timeout 20000 for client /127.0.0.1:36572 [junit4:junit4] 2> 1909 T421 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230008, negotiated timeout = 20000 [junit4:junit4] 2> 1909 T422 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@85d6f2 name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 1910 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 1911 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230008 type:create cxid:0x1 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 1912 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230008 type:create cxid:0x3 zxid:0x39 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 1918 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4:junit4] 2> 2922 T368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51183_iw_ [junit4:junit4] 2> 2923 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230008 type:delete cxid:0xd zxid:0x3a txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:51183_iw_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:51183_iw_ [junit4:junit4] 2> 2925 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51183_iw_ [junit4:junit4] 2> 2932 T390 oascc.ZkStateReader$3.process Updating live nodes... (3) [junit4:junit4] 2> 2933 T395 oascc.ZkStateReader$3.process Updating live nodes... (3) [junit4:junit4] 2> 2934 T408 oascc.ZkStateReader$3.process Updating live nodes... (3) [junit4:junit4] 2> 2935 T422 oascc.ZkStateReader$3.process Updating live nodes... (3) [junit4:junit4] 2> 2946 T368 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0 [junit4:junit4] 2> 2948 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4:junit4] 2> 2949 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 3045 T368 oejs.Server.doStart jetty-8.1.8.v20121106 [junit4:junit4] 2> 3048 T368 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55443 [junit4:junit4] 2> 3048 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4:junit4] 2> 3049 T368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4:junit4] 2> 3049 T368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026 [junit4:junit4] 2> 3050 T368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/solr.xml [junit4:junit4] 2> 3050 T368 oasc.CoreContainer. New CoreContainer 18644155 [junit4:junit4] 2> 3050 T368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/' [junit4:junit4] 2> 3051 T368 oasc.SolrResourceLoader. new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/' [junit4:junit4] 2> 3073 T368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000 [junit4:junit4] 2> 3073 T368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http:// [junit4:junit4] 2> 3074 T368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4:junit4] 2> 3074 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4:junit4] 2> 3074 T368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4:junit4] 2> 3075 T368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4:junit4] 2> 3075 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4:junit4] 2> 3076 T368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4:junit4] 2> 3076 T368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4:junit4] 2> 3076 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false [junit4:junit4] 2> 3085 T368 oasc.CoreContainer.load Registering Log Listener [junit4:junit4] 2> 3101 T368 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53441/solr [junit4:junit4] 2> 3102 T368 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4:junit4] 2> 3102 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@17fc368 [junit4:junit4] 2> 3103 T433 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 3108 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 3120 T433 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 3120 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36573 [junit4:junit4] 2> 3121 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36573 [junit4:junit4] 2> 3121 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee55230009 with negotiated timeout 20000 for client /127.0.0.1:36573 [junit4:junit4] 2> 3122 T433 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee55230009, negotiated timeout = 20000 [junit4:junit4] 2> 3122 T434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17fc368 name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 3123 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 3124 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee55230009 [junit4:junit4] 2> 3125 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36573 which had sessionid 0x13d8fee55230009 [junit4:junit4] 2> 3125 T434 oaz.ClientCnxn$EventThread.run EventThread shut down [junit4:junit4] 2> 3125 T368 oaz.ZooKeeper.close Session: 0x13d8fee55230009 closed [junit4:junit4] 2> 3126 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000 [junit4:junit4] 2> 3130 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@939704 [junit4:junit4] 2> 3131 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 3132 T435 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 3132 T435 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 3132 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36574 [junit4:junit4] 2> 3133 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36574 [junit4:junit4] 2> 3133 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee5523000a with negotiated timeout 20000 for client /127.0.0.1:36574 [junit4:junit4] 2> 3133 T435 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee5523000a, negotiated timeout = 20000 [junit4:junit4] 2> 3134 T436 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@939704 name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 3134 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 3135 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000a type:create cxid:0x1 zxid:0x40 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 3136 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000a type:create cxid:0x3 zxid:0x41 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 3141 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4:junit4] 2> 4144 T368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55443_iw_ [junit4:junit4] 2> 4145 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000a type:delete cxid:0xd zxid:0x42 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:55443_iw_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:55443_iw_ [junit4:junit4] 2> 4146 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55443_iw_ [junit4:junit4] 2> 4149 T395 oascc.ZkStateReader$3.process Updating live nodes... (4) [junit4:junit4] 2> 4149 T390 oascc.ZkStateReader$3.process Updating live nodes... (4) [junit4:junit4] 2> 4149 T408 oascc.ZkStateReader$3.process Updating live nodes... (4) [junit4:junit4] 2> 4149 T422 oascc.ZkStateReader$3.process Updating live nodes... (4) [junit4:junit4] 2> 4149 T436 oascc.ZkStateReader$3.process Updating live nodes... (4) [junit4:junit4] 2> 4154 T368 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0 [junit4:junit4] 2> 4155 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4:junit4] 2> 4155 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 4234 T368 oejs.Server.doStart jetty-8.1.8.v20121106 [junit4:junit4] 2> 4236 T368 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59877 [junit4:junit4] 2> 4237 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4:junit4] 2> 4237 T368 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4:junit4] 2> 4237 T368 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1363919398229 [junit4:junit4] 2> 4238 T368 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1363919398229/solr.xml [junit4:junit4] 2> 4238 T368 oasc.CoreContainer. New CoreContainer 31507358 [junit4:junit4] 2> 4239 T368 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1363919398229/' [junit4:junit4] 2> 4239 T368 oasc.SolrResourceLoader. new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1363919398229/' [junit4:junit4] 2> 4262 T368 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000 [junit4:junit4] 2> 4263 T368 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http:// [junit4:junit4] 2> 4263 T368 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4:junit4] 2> 4264 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4:junit4] 2> 4264 T368 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4:junit4] 2> 4265 T368 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4:junit4] 2> 4265 T368 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4:junit4] 2> 4266 T368 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4:junit4] 2> 4266 T368 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4:junit4] 2> 4266 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false [junit4:junit4] 2> 4274 T368 oasc.CoreContainer.load Registering Log Listener [junit4:junit4] 2> 4289 T368 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:53441/solr [junit4:junit4] 2> 4289 T368 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4:junit4] 2> 4290 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3846a3 [junit4:junit4] 2> 4291 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 4292 T447 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 4293 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36575 [junit4:junit4] 2> 4293 T447 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 4293 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36575 [junit4:junit4] 2> 4294 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee5523000b with negotiated timeout 20000 for client /127.0.0.1:36575 [junit4:junit4] 2> 4294 T447 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee5523000b, negotiated timeout = 20000 [junit4:junit4] 2> 4295 T448 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3846a3 name:ZooKeeperConnection Watcher:127.0.0.1:53441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 4295 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 4297 T373 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8fee5523000b [junit4:junit4] 2> 4297 T370 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:36575 which had sessionid 0x13d8fee5523000b [junit4:junit4] 2> 4297 T368 oaz.ZooKeeper.close Session: 0x13d8fee5523000b closed [junit4:junit4] 2> 4297 T448 oaz.ClientCnxn$EventThread.run EventThread shut down [junit4:junit4] 2> 4298 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000 [junit4:junit4] 2> 4302 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@19e97be [junit4:junit4] 2> 4303 T449 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 4303 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 4304 T449 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 4304 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36576 [junit4:junit4] 2> 4305 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36576 [junit4:junit4] 2> 4306 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee5523000c with negotiated timeout 20000 for client /127.0.0.1:36576 [junit4:junit4] 2> 4306 T449 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee5523000c, negotiated timeout = 20000 [junit4:junit4] 2> 4306 T450 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19e97be name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 4306 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 4308 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000c type:create cxid:0x1 zxid:0x48 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 4309 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000c type:create cxid:0x3 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 4311 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4:junit4] 2> 5315 T368 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59877_iw_ [junit4:junit4] 2> 5315 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee5523000c type:delete cxid:0xd zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:59877_iw_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:59877_iw_ [junit4:junit4] 2> 5316 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59877_iw_ [junit4:junit4] 2> 5319 T390 oascc.ZkStateReader$3.process Updating live nodes... (5) [junit4:junit4] 2> 5319 T450 oascc.ZkStateReader$3.process Updating live nodes... (5) [junit4:junit4] 2> 5319 T436 oascc.ZkStateReader$3.process Updating live nodes... (5) [junit4:junit4] 2> 5319 T395 oascc.ZkStateReader$3.process Updating live nodes... (5) [junit4:junit4] 2> 5319 T422 oascc.ZkStateReader$3.process Updating live nodes... (5) [junit4:junit4] 2> 5319 T408 oascc.ZkStateReader$3.process Updating live nodes... (5) [junit4:junit4] 2> 5324 T368 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0 [junit4:junit4] 2> 5325 T368 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4:junit4] 2> 5325 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 5330 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 5341 T404 oasc.CoreContainer.create Creating SolrCore 'unloadcollection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/unloadcollection1 [junit4:junit4] 2> 5342 T404 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection [junit4:junit4] 2> 5342 T404 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:unloadcollection [junit4:junit4] 2> 5343 T404 oasc.ZkController.getConfName Looking for collection configName [junit4:junit4] 2> 5343 T404 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1 [junit4:junit4] 2> 5344 T404 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection [junit4:junit4] 2> 5346 T404 oasc.ZkController.readConfigName Load collection config from:/collections/unloadcollection [junit4:junit4] 2> 5347 T404 oasc.SolrResourceLoader. new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/unloadcollection1/' [junit4:junit4] 2> 5379 T404 oasc.SolrConfig. Using Lucene MatchVersion: LUCENE_50 [junit4:junit4] 2> 5424 T404 oasc.SolrConfig. Loaded SolrConfig: solrconfig.xml [junit4:junit4] 2> 5425 T404 oass.IndexSchema.readSchema Reading Solr Schema [junit4:junit4] 2> 5430 T404 oass.IndexSchema.readSchema Schema name=test [junit4:junit4] 2> 5921 T404 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4:junit4] 2> 5928 T404 oass.IndexSchema.readSchema default search field in schema is text [junit4:junit4] 2> 5932 T404 oass.IndexSchema.readSchema unique key field: id [junit4:junit4] 2> 5944 T404 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4:junit4] 2> 5948 T404 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4:junit4] 2> 5952 T404 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4:junit4] 2> 5953 T404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Unknown key IMPORTANT NOTE [junit4:junit4] 2> 5954 T404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Expected key, got STRING [junit4:junit4] 2> 5954 T404 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4:junit4] 2> 5955 T404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Unknown key IMPORTANT NOTE [junit4:junit4] 2> 5956 T404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Expected key, got STRING [junit4:junit4] 2> 5956 T404 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 2> 5956 T404 oasc.SolrCore. [unloadcollection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/unloadcollection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n/ [junit4:junit4] 2> 5957 T404 oasc.JmxMonitoredMap. JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@143f408 [junit4:junit4] 2> 5958 T404 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 2> 5958 T404 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n [junit4:junit4] 2> 5959 T404 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n/index/ [junit4:junit4] 2> 5959 T404 oasc.SolrCore.initIndex WARNING [unloadcollection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n/index' doesn't exist. Creating new index... [junit4:junit4] 2> 5960 T404 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n/index [junit4:junit4] 2> 5961 T404 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1 [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1] [junit4:junit4] 2> 5962 T404 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1] [junit4:junit4] 2> 5974 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 2> 5974 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 2> 5975 T404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4:junit4] 2> 5975 T404 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4:junit4] 2> 5976 T404 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler [junit4:junit4] 2> 5976 T404 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 2> 5976 T404 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4:junit4] 2> 5977 T404 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4:junit4] 2> 5977 T404 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers [junit4:junit4] 2> 5980 T404 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4:junit4] 2> 5983 T404 oass.SolrIndexSearcher. Opening Searcher@1d77c39 main [junit4:junit4] 2> 5984 T404 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n/tlog [junit4:junit4] 2> 5987 T404 oasu.CommitTracker. Hard AutoCommit: disabled [junit4:junit4] 2> 5987 T404 oasu.CommitTracker. Soft AutoCommit: disabled [junit4:junit4] 2> 5991 T452 oasc.SolrCore.registerSearcher [unloadcollection1] Registered new searcher Searcher@1d77c39 main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 2> 5992 T404 oasc.ZkController.publish publishing core=unloadcollection1 state=down [junit4:junit4] 2> 6527 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 6528 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={ [junit4:junit4] 2> "core":"unloadcollection1", [junit4:junit4] 2> "numShards":"1", [junit4:junit4] 2> "collection":"unloadcollection", [junit4:junit4] 2> "node_name":"127.0.0.1:47486_iw_", [junit4:junit4] 2> "core_node_name":null, [junit4:junit4] 2> "shard":null, [junit4:junit4] 2> "operation":"state", [junit4:junit4] 2> "base_url":"http://127.0.0.1:47486/iw_", [junit4:junit4] 2> "roles":null, [junit4:junit4] 2> "state":"down"} [junit4:junit4] 2> 6528 T391 oasc.Overseer$ClusterStateUpdater.createCollection Create collection unloadcollection with numShards 1 [junit4:junit4] 2> 6537 T391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1 [junit4:junit4] 2> 6543 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 6543 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 6543 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 6543 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 6543 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 6543 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 6993 T404 oasc.CoreContainer.registerCore registering core: unloadcollection1 [junit4:junit4] 2> 6994 T404 oasc.ZkController.register Register replica - core:unloadcollection1 address:http://127.0.0.1:47486/iw_ collection:unloadcollection shard:shard1 [junit4:junit4] 2> 7001 T404 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false [junit4:junit4] 2> 7012 T404 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection/leader_elect/shard1/election [junit4:junit4] 2> 7021 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:delete cxid:0x51 zxid:0x57 txntype:-1 reqpath:n/a Error Path:/solr/collections/unloadcollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/unloadcollection/leaders [junit4:junit4] 2> 7022 T404 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process. [junit4:junit4] 2> 7023 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:create cxid:0x52 zxid:0x58 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 7026 T404 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4:junit4] 2> 7026 T404 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4:junit4] 2> 7026 T404 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47486/iw_/unloadcollection1/ [junit4:junit4] 2> 7027 T404 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4:junit4] 2> 7027 T404 oasc.SyncStrategy.syncToMe http://127.0.0.1:47486/iw_/unloadcollection1/ has no replicas [junit4:junit4] 2> 7028 T404 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47486/iw_/unloadcollection1/ [junit4:junit4] 2> 7028 T404 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection/leaders/shard1 [junit4:junit4] 2> 7032 T373 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8fee55230006 type:create cxid:0x5d zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer [junit4:junit4] 2> 8048 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 8054 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 8054 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 8054 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 8054 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 8054 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 8054 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 8086 T404 oasc.ZkController.register We are http://127.0.0.1:47486/iw_/unloadcollection1/ and leader is http://127.0.0.1:47486/iw_/unloadcollection1/ [junit4:junit4] 2> 8088 T404 oasc.ZkController.register No LogReplay needed for core=unloadcollection1 baseURL=http://127.0.0.1:47486/iw_ [junit4:junit4] 2> 8088 T404 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4:junit4] 2> 8089 T404 oasc.ZkController.publish publishing core=unloadcollection1 state=active [junit4:junit4] 2> 8090 T404 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 8091 T404 oasc.CoreContainer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/solr.xml [junit4:junit4] 2> 8099 T404 oasc.SolrXMLSerializer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1363919394607/solr.xml [junit4:junit4] 2> 8100 T404 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {action=CREATE&version=2&numShards=1&collection=unloadcollection&name=unloadcollection1&wt=javabin&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919399407unloadcollection1_1n} status=0 QTime=2760 [junit4:junit4] 2> 8100 T368 oascsi.HttpClientUtil.createClient Creating new http client, config: [junit4:junit4] 2> 8102 T368 oaz.ZooKeeper. Initiating client connection, connectString=127.0.0.1:53441/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1550e0e [junit4:junit4] 2> 8103 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4:junit4] 2> 8103 T453 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:53441. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration")) [junit4:junit4] 2> 8104 T453 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:53441, initiating session [junit4:junit4] 2> 8104 T370 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:36578 [junit4:junit4] 2> 8104 T370 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:36578 [junit4:junit4] 2> 8106 T372 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8fee5523000d with negotiated timeout 10000 for client /127.0.0.1:36578 [junit4:junit4] 2> 8106 T453 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:53441, sessionid = 0x13d8fee5523000d, negotiated timeout = 10000 [junit4:junit4] 2> 8106 T454 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1550e0e name:ZooKeeperConnection Watcher:127.0.0.1:53441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 2> 8107 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4:junit4] 2> 8108 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4:junit4] 2> 8110 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 8111 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 8116 T415 oasc.CoreContainer.create Creating SolrCore 'unloadcollection2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/unloadcollection2 [junit4:junit4] 2> 8117 T415 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection [junit4:junit4] 2> 8117 T415 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4:junit4] 2> 8118 T415 oasc.ZkController.readConfigName Load collection config from:/collections/unloadcollection [junit4:junit4] 2> 8119 T415 oasc.SolrResourceLoader. new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/unloadcollection2/' [junit4:junit4] 2> 8162 T415 oasc.SolrConfig. Using Lucene MatchVersion: LUCENE_50 [junit4:junit4] 2> 8209 T415 oasc.SolrConfig. Loaded SolrConfig: solrconfig.xml [junit4:junit4] 2> 8210 T415 oass.IndexSchema.readSchema Reading Solr Schema [junit4:junit4] 2> 8215 T415 oass.IndexSchema.readSchema Schema name=test [junit4:junit4] 2> 8690 T415 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4:junit4] 2> 8700 T415 oass.IndexSchema.readSchema default search field in schema is text [junit4:junit4] 2> 8705 T415 oass.IndexSchema.readSchema unique key field: id [junit4:junit4] 2> 8721 T415 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4:junit4] 2> 8725 T415 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4:junit4] 2> 8729 T415 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4:junit4] 2> 8731 T415 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Unknown key IMPORTANT NOTE [junit4:junit4] 2> 8732 T415 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Expected key, got STRING [junit4:junit4] 2> 8732 T415 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4:junit4] 2> 8733 T415 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Unknown key IMPORTANT NOTE [junit4:junit4] 2> 8733 T415 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Expected key, got STRING [junit4:junit4] 2> 8734 T415 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 2> 8734 T415 oasc.SolrCore. [unloadcollection2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/unloadcollection2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/ [junit4:junit4] 2> 8734 T415 oasc.JmxMonitoredMap. JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@143f408 [junit4:junit4] 2> 8735 T415 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 2> 8735 T415 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n [junit4:junit4] 2> 8736 T415 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/index/ [junit4:junit4] 2> 8736 T415 oasc.SolrCore.initIndex WARNING [unloadcollection2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/index' doesn't exist. Creating new index... [junit4:junit4] 2> 8736 T415 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/index [junit4:junit4] 2> 8738 T415 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1 [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@c1f890 lockFactory=org.apache.lucene.store.NativeFSLockFactory@32ce9e; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1] [junit4:junit4] 2> 8738 T415 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1] [junit4:junit4] 2> 8740 T415 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 2> 8740 T415 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 2> 8740 T415 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4:junit4] 2> 8741 T415 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4:junit4] 2> 8741 T415 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler [junit4:junit4] 2> 8742 T415 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 2> 8742 T415 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4:junit4] 2> 8743 T415 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4:junit4] 2> 8743 T415 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers [junit4:junit4] 2> 8746 T415 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4:junit4] 2> 8750 T415 oass.SolrIndexSearcher. Opening Searcher@140cc01 main [junit4:junit4] 2> 8750 T415 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/tlog [junit4:junit4] 2> 8751 T415 oasu.CommitTracker. Hard AutoCommit: disabled [junit4:junit4] 2> 8751 T415 oasu.CommitTracker. Soft AutoCommit: disabled [junit4:junit4] 2> 8755 T455 oasc.SolrCore.registerSearcher [unloadcollection2] Registered new searcher Searcher@140cc01 main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 2> 8756 T415 oasc.ZkController.publish publishing core=unloadcollection2 state=down [junit4:junit4] 2> 8756 T415 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4:junit4] 2> 9560 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 9561 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={ [junit4:junit4] 2> "core":"unloadcollection1", [junit4:junit4] 2> "numShards":"1", [junit4:junit4] 2> "collection":"unloadcollection", [junit4:junit4] 2> "node_name":"127.0.0.1:47486_iw_", [junit4:junit4] 2> "core_node_name":"127.0.0.1:47486_iw__unloadcollection1", [junit4:junit4] 2> "shard":"shard1", [junit4:junit4] 2> "operation":"state", [junit4:junit4] 2> "base_url":"http://127.0.0.1:47486/iw_", [junit4:junit4] 2> "roles":null, [junit4:junit4] 2> "state":"active"} [junit4:junit4] 2> 9565 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4:junit4] 2> "core":"unloadcollection2", [junit4:junit4] 2> "numShards":"2", [junit4:junit4] 2> "collection":"unloadcollection", [junit4:junit4] 2> "node_name":"127.0.0.1:51183_iw_", [junit4:junit4] 2> "core_node_name":null, [junit4:junit4] 2> "shard":null, [junit4:junit4] 2> "operation":"state", [junit4:junit4] 2> "base_url":"http://127.0.0.1:51183/iw_", [junit4:junit4] 2> "roles":null, [junit4:junit4] 2> "state":"down"} [junit4:junit4] 2> 9565 T391 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1 [junit4:junit4] 2> 9566 T391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1 [junit4:junit4] 2> 9569 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 9569 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 9569 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 9569 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 9569 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 9569 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 9569 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 9758 T415 oasc.CoreContainer.registerCore registering core: unloadcollection2 [junit4:junit4] 2> 9758 T415 oasc.ZkController.register Register replica - core:unloadcollection2 address:http://127.0.0.1:51183/iw_ collection:unloadcollection shard:shard1 [junit4:junit4] 2> 9764 T415 oasc.ZkController.register We are http://127.0.0.1:51183/iw_/unloadcollection2/ and leader is http://127.0.0.1:47486/iw_/unloadcollection1/ [junit4:junit4] 2> 9764 T415 oasc.ZkController.register No LogReplay needed for core=unloadcollection2 baseURL=http://127.0.0.1:51183/iw_ [junit4:junit4] 2> 9765 T415 oasc.ZkController.checkRecovery Core needs to recover:unloadcollection2 [junit4:junit4] 2> 9765 T415 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery [junit4:junit4] 2> ASYNC NEW_CORE C25 name=unloadcollection2 org.apache.solr.core.SolrCore@1467e93 url=http://127.0.0.1:51183/iw_/unloadcollection2 node=127.0.0.1:51183_iw_ C25_STATE=coll:unloadcollection core:unloadcollection2 props:{core=unloadcollection2, collection=unloadcollection, node_name=127.0.0.1:51183_iw_, base_url=http://127.0.0.1:51183/iw_, state=down} [junit4:junit4] 2> 9773 T456 C25 P51183 oasc.RecoveryStrategy.run Starting recovery process. core=unloadcollection2 recoveringAfterStartup=true [junit4:junit4] 2> 9774 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[] [junit4:junit4] 2> 9775 T456 C25 P51183 oasc.ZkController.publish publishing core=unloadcollection2 state=recovering [junit4:junit4] 2> 9775 T456 C25 P51183 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4:junit4] 2> 9773 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 9777 T456 C25 P51183 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 9777 T415 oasc.CoreContainer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/solr.xml [junit4:junit4] 2> 9789 T415 oasc.SolrXMLSerializer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1363919395792/solr.xml [junit4:junit4] 2> 9790 T415 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {action=CREATE&version=2&collection=unloadcollection&name=unloadcollection2&wt=javabin&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n} status=0 QTime=1674 [junit4:junit4] 2> 9792 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 9793 T368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: unloadcollection failOnTimeout:true timeout (sec):230 [junit4:junit4] 2> 9793 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 10795 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 11076 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 11077 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4:junit4] 2> "core":"unloadcollection2", [junit4:junit4] 2> "numShards":"2", [junit4:junit4] 2> "collection":"unloadcollection", [junit4:junit4] 2> "node_name":"127.0.0.1:51183_iw_", [junit4:junit4] 2> "core_node_name":"127.0.0.1:51183_iw__unloadcollection2", [junit4:junit4] 2> "shard":"shard1", [junit4:junit4] 2> "operation":"state", [junit4:junit4] 2> "base_url":"http://127.0.0.1:51183/iw_", [junit4:junit4] 2> "roles":null, [junit4:junit4] 2> "state":"recovering"} [junit4:junit4] 2> 11087 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 11088 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 11088 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 11088 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 11088 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 11088 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 11091 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 11784 T401 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:51183_iw__unloadcollection2&core=unloadcollection1&nodeName=127.0.0.1:51183_iw_&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&wt=javabin&state=recovering} status=0 QTime=2000 [junit4:junit4] 2> 11798 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 12799 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> C25_STATE=coll:unloadcollection core:unloadcollection2 props:{core=unloadcollection2, collection=unloadcollection, node_name=127.0.0.1:51183_iw_, shard=shard1, base_url=http://127.0.0.1:51183/iw_, state=recovering} [junit4:junit4] 2> 13785 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:47486/iw_/unloadcollection1/ core=unloadcollection2 - recoveringAfterStartup=true [junit4:junit4] 2> 13791 T456 C25 P51183 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false [junit4:junit4] 2> 13798 T456 C25 P51183 oasu.PeerSync.sync PeerSync: core=unloadcollection2 url=http://127.0.0.1:51183/iw_ START replicas=[http://127.0.0.1:47486/iw_/unloadcollection1/] nUpdates=100 [junit4:junit4] 2> 13799 T456 C25 P51183 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates [junit4:junit4] 2> 13807 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=unloadcollection2 [junit4:junit4] 2> 13807 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=unloadcollection2 [junit4:junit4] 2> 13807 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=unloadcollection2 [junit4:junit4] 2> 13808 T456 C25 P51183 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} [junit4:junit4] 2> 13808 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 13808 T456 C25 P51183 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:47486/iw_/unloadcollection1/. core=unloadcollection2 [junit4:junit4] 2> 13808 T456 C25 P51183 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> ASYNC NEW_CORE C26 name=unloadcollection1 org.apache.solr.core.SolrCore@38adab url=http://127.0.0.1:47486/iw_/unloadcollection1 node=127.0.0.1:47486_iw_ C26_STATE=coll:unloadcollection core:unloadcollection1 props:{core=unloadcollection1, collection=unloadcollection, node_name=127.0.0.1:47486_iw_, shard=shard1, base_url=http://127.0.0.1:47486/iw_, state=active, leader=true} [junit4:junit4] 2> 13813 T402 C26 P47486 REQ /get {getVersions=100&version=2&distrib=false&qt=/get&wt=javabin} status=0 QTime=1 [junit4:junit4] 2> 13822 T404 C26 P47486 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} [junit4:junit4] 2> 13824 T404 C26 P47486 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1 [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1] [junit4:junit4] 2> 13824 T404 C26 P47486 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1] [junit4:junit4] 2> 13825 T404 C26 P47486 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2 [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1] [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_2,generation=2,filenames=[segments_2] [junit4:junit4] 2> 13825 T404 C26 P47486 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2] [junit4:junit4] 2> 13826 T404 C26 P47486 oass.SolrIndexSearcher. Opening Searcher@122430c realtime [junit4:junit4] 2> 13826 T404 C26 P47486 oasu.DirectUpdateHandler2.commit end_commit_flush [junit4:junit4] 2> 13827 T404 C26 P47486 /update {waitSearcher=true&softCommit=false&version=2&wt=javabin&commit_end_point=true&commit=true&openSearcher=false} {commit=} 0 11 [junit4:junit4] 2> 13857 T456 C25 P51183 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4:junit4] 2> 13857 T456 C25 P51183 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000 [junit4:junit4] 2> 13864 T456 C25 P51183 oash.SnapPuller. No value set for 'pollInterval'. Timer Task not started. [junit4:junit4] 2> 13873 T398 C26 P47486 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4:junit4] 2> 13874 T398 C26 P47486 REQ /replication {version=2&qt=/replication&command=indexversion&wt=javabin} status=0 QTime=8 [junit4:junit4] 2> 13876 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Master's generation: 2 [junit4:junit4] 2> 13876 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Slave's generation: 1 [junit4:junit4] 2> 13877 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Starting replication process [junit4:junit4] 2> 13879 T401 C26 P47486 REQ /replication {version=2&qt=/replication&command=filelist&wt=javabin&generation=2} status=0 QTime=0 [junit4:junit4] 2> 13880 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1 [junit4:junit4] 2> 13881 T456 C25 P51183 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919402185unloadcollection1_2n/index.20130322033007950 [junit4:junit4] 2> 13882 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Starting download to BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6b345c lockFactory=org.apache.lucene.store.NativeFSLockFactory@92b992) fullCopy=false [junit4:junit4] 2> 13890 T402 C26 P47486 REQ /replication {checksum=true&qt=/replication&command=filecontent&wt=filestream&file=segments_2&generation=2} status=0 QTime=3 [junit4:junit4] 2> 13897 T456 C25 P51183 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs [junit4:junit4] 2> 13899 T456 C25 P51183 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter... [junit4:junit4] 2> 13899 T456 C25 P51183 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=unloadcollection2 [junit4:junit4] 2> 13900 T456 C25 P51183 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2 [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@c1f890 lockFactory=org.apache.lucene.store.NativeFSLockFactory@32ce9e; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1] [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@c1f890 lockFactory=org.apache.lucene.store.NativeFSLockFactory@32ce9e; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_2,generation=2,filenames=[segments_2] [junit4:junit4] 2> 13901 T456 C25 P51183 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2] [junit4:junit4] 2> 13901 T456 C25 P51183 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used. [junit4:junit4] 2> 13901 T456 C25 P51183 oass.SolrIndexSearcher. Opening Searcher@1d529e3 main [junit4:junit4] 2> 13902 T455 oasc.SolrCore.registerSearcher [unloadcollection2] Registered new searcher Searcher@1d529e3 main{StandardDirectoryReader(segments_2:1:nrt)} [junit4:junit4] 2> 13903 T456 C25 P51183 oasc.RecoveryStrategy.replay No replay needed. core=unloadcollection2 [junit4:junit4] 2> 13903 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=unloadcollection2 [junit4:junit4] 2> 13903 T456 C25 P51183 oasc.ZkController.publish publishing core=unloadcollection2 state=active [junit4:junit4] 2> 13904 T456 C25 P51183 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4:junit4] 2> 13905 T456 C25 P51183 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=unloadcollection2 [junit4:junit4] 2> 14097 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 14098 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4:junit4] 2> "core":"unloadcollection2", [junit4:junit4] 2> "numShards":"2", [junit4:junit4] 2> "collection":"unloadcollection", [junit4:junit4] 2> "node_name":"127.0.0.1:51183_iw_", [junit4:junit4] 2> "core_node_name":"127.0.0.1:51183_iw__unloadcollection2", [junit4:junit4] 2> "shard":"shard1", [junit4:junit4] 2> "operation":"state", [junit4:junit4] 2> "base_url":"http://127.0.0.1:51183/iw_", [junit4:junit4] 2> "roles":null, [junit4:junit4] 2> "state":"active"} [junit4:junit4] 2> 14102 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 14102 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 14102 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 14102 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 14102 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 14103 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 14102 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 14813 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 14814 T368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: unloadcollection [junit4:junit4] 2> 14814 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 14819 T431 oasc.CoreContainer.create Creating SolrCore 'unloadcollection3' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/unloadcollection3 [junit4:junit4] 2> 14820 T431 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection [junit4:junit4] 2> 14820 T431 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4:junit4] 2> 14821 T431 oasc.ZkController.readConfigName Load collection config from:/collections/unloadcollection [junit4:junit4] 2> 14822 T431 oasc.SolrResourceLoader. new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/unloadcollection3/' [junit4:junit4] 2> 14862 T431 oasc.SolrConfig. Using Lucene MatchVersion: LUCENE_50 [junit4:junit4] 2> 14917 T431 oasc.SolrConfig. Loaded SolrConfig: solrconfig.xml [junit4:junit4] 2> 14918 T431 oass.IndexSchema.readSchema Reading Solr Schema [junit4:junit4] 2> 14925 T431 oass.IndexSchema.readSchema Schema name=test [junit4:junit4] 2> 15445 T431 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4:junit4] 2> 15452 T431 oass.IndexSchema.readSchema default search field in schema is text [junit4:junit4] 2> 15457 T431 oass.IndexSchema.readSchema unique key field: id [junit4:junit4] 2> 15468 T431 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4:junit4] 2> 15472 T431 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4:junit4] 2> 15475 T431 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4:junit4] 2> 15477 T431 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Unknown key IMPORTANT NOTE [junit4:junit4] 2> 15477 T431 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Expected key, got STRING [junit4:junit4] 2> 15478 T431 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4:junit4] 2> 15479 T431 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Unknown key IMPORTANT NOTE [junit4:junit4] 2> 15479 T431 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates. WARNING Expected key, got STRING [junit4:junit4] 2> 15480 T431 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 2> 15480 T431 oasc.SolrCore. [unloadcollection3] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/unloadcollection3/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/ [junit4:junit4] 2> 15480 T431 oasc.JmxMonitoredMap. JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@143f408 [junit4:junit4] 2> 15481 T431 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 2> 15482 T431 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n [junit4:junit4] 2> 15482 T431 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/index/ [junit4:junit4] 2> 15482 T431 oasc.SolrCore.initIndex WARNING [unloadcollection3] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/index' doesn't exist. Creating new index... [junit4:junit4] 2> 15483 T431 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/index [junit4:junit4] 2> 15485 T431 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1 [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@16b45d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6af33; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1] [junit4:junit4] 2> 15485 T431 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1] [junit4:junit4] 2> 15487 T431 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 2> 15487 T431 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 2> 15488 T431 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4:junit4] 2> 15488 T431 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4:junit4] 2> 15489 T431 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler [junit4:junit4] 2> 15489 T431 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 2> 15490 T431 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4:junit4] 2> 15490 T431 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4:junit4] 2> 15491 T431 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers [junit4:junit4] 2> 15494 T431 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4:junit4] 2> 15497 T431 oass.SolrIndexSearcher. Opening Searcher@15b6a1e main [junit4:junit4] 2> 15498 T431 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/tlog [junit4:junit4] 2> 15498 T431 oasu.CommitTracker. Hard AutoCommit: disabled [junit4:junit4] 2> 15499 T431 oasu.CommitTracker. Soft AutoCommit: disabled [junit4:junit4] 2> 15504 T459 oasc.SolrCore.registerSearcher [unloadcollection3] Registered new searcher Searcher@15b6a1e main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 2> 15504 T431 oasc.ZkController.publish publishing core=unloadcollection3 state=down [junit4:junit4] 2> 15505 T431 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4:junit4] 2> 15609 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 15611 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4:junit4] 2> "core":"unloadcollection3", [junit4:junit4] 2> "numShards":"2", [junit4:junit4] 2> "collection":"unloadcollection", [junit4:junit4] 2> "node_name":"127.0.0.1:55443_iw_", [junit4:junit4] 2> "core_node_name":null, [junit4:junit4] 2> "shard":null, [junit4:junit4] 2> "operation":"state", [junit4:junit4] 2> "base_url":"http://127.0.0.1:55443/iw_", [junit4:junit4] 2> "roles":null, [junit4:junit4] 2> "state":"down"} [junit4:junit4] 2> 15611 T391 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1 [junit4:junit4] 2> 15611 T391 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1 [junit4:junit4] 2> 15618 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 15618 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 15618 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 15618 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 15618 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 15619 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 15618 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 16506 T431 oasc.CoreContainer.registerCore registering core: unloadcollection3 [junit4:junit4] 2> 16507 T431 oasc.ZkController.register Register replica - core:unloadcollection3 address:http://127.0.0.1:55443/iw_ collection:unloadcollection shard:shard1 [junit4:junit4] 2> 16512 T431 oasc.ZkController.register We are http://127.0.0.1:55443/iw_/unloadcollection3/ and leader is http://127.0.0.1:47486/iw_/unloadcollection1/ [junit4:junit4] 2> 16513 T431 oasc.ZkController.register No LogReplay needed for core=unloadcollection3 baseURL=http://127.0.0.1:55443/iw_ [junit4:junit4] 2> 16514 T431 oasc.ZkController.checkRecovery Core needs to recover:unloadcollection3 [junit4:junit4] 2> 16515 T431 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery [junit4:junit4] 2> ASYNC NEW_CORE C27 name=unloadcollection3 org.apache.solr.core.SolrCore@12d0b21 url=http://127.0.0.1:55443/iw_/unloadcollection3 node=127.0.0.1:55443_iw_ C27_STATE=coll:unloadcollection core:unloadcollection3 props:{core=unloadcollection3, collection=unloadcollection, node_name=127.0.0.1:55443_iw_, base_url=http://127.0.0.1:55443/iw_, state=down} [junit4:junit4] 2> 16515 T460 C27 P55443 oasc.RecoveryStrategy.run Starting recovery process. core=unloadcollection3 recoveringAfterStartup=true [junit4:junit4] 2> 16516 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[] [junit4:junit4] 2> 16517 T460 C27 P55443 oasc.ZkController.publish publishing core=unloadcollection3 state=recovering [junit4:junit4] 2> 16517 T460 C27 P55443 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4:junit4] 2> 16517 T431 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 16518 T460 C27 P55443 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 16518 T431 oasc.CoreContainer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/solr.xml [junit4:junit4] 2> 16528 T431 oasc.SolrXMLSerializer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1363919397026/solr.xml [junit4:junit4] 2> 16529 T431 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {action=CREATE&version=2&collection=unloadcollection&name=unloadcollection3&wt=javabin&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n} status=0 QTime=1710 [junit4:junit4] 2> 16530 T368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: unloadcollection failOnTimeout:true timeout (sec):230 [junit4:junit4] 2> 16531 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 17124 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 17125 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4:junit4] 2> "core":"unloadcollection3", [junit4:junit4] 2> "numShards":"2", [junit4:junit4] 2> "collection":"unloadcollection", [junit4:junit4] 2> "node_name":"127.0.0.1:55443_iw_", [junit4:junit4] 2> "core_node_name":"127.0.0.1:55443_iw__unloadcollection3", [junit4:junit4] 2> "shard":"shard1", [junit4:junit4] 2> "operation":"state", [junit4:junit4] 2> "base_url":"http://127.0.0.1:55443/iw_", [junit4:junit4] 2> "roles":null, [junit4:junit4] 2> "state":"recovering"} [junit4:junit4] 2> 17128 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 17128 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 17128 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 17128 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 17128 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 17128 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 17128 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 17525 T398 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:55443_iw__unloadcollection3&core=unloadcollection1&nodeName=127.0.0.1:55443_iw_&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&wt=javabin&state=recovering} status=0 QTime=1000 [junit4:junit4] 2> 17532 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 18534 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> C27_STATE=coll:unloadcollection core:unloadcollection3 props:{core=unloadcollection3, collection=unloadcollection, node_name=127.0.0.1:55443_iw_, shard=shard1, base_url=http://127.0.0.1:55443/iw_, state=recovering} [junit4:junit4] 2> 19526 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:47486/iw_/unloadcollection1/ core=unloadcollection3 - recoveringAfterStartup=true [junit4:junit4] 2> 19526 T460 C27 P55443 oasu.PeerSync.sync PeerSync: core=unloadcollection3 url=http://127.0.0.1:55443/iw_ START replicas=[http://127.0.0.1:47486/iw_/unloadcollection1/] nUpdates=100 [junit4:junit4] 2> 19527 T460 C27 P55443 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates [junit4:junit4] 2> 19527 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=unloadcollection3 [junit4:junit4] 2> 19527 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=unloadcollection3 [junit4:junit4] 2> 19527 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=unloadcollection3 [junit4:junit4] 2> 19527 T460 C27 P55443 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} [junit4:junit4] 2> 19527 T460 C27 P55443 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:47486/iw_/unloadcollection1/. core=unloadcollection3 [junit4:junit4] 2> 19527 T460 C27 P55443 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> 19529 T404 C26 P47486 REQ /get {getVersions=100&version=2&distrib=false&qt=/get&wt=javabin} status=0 QTime=0 [junit4:junit4] 2> 19533 T401 C26 P47486 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} [junit4:junit4] 2> 19534 T401 C26 P47486 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2 [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_2,generation=2,filenames=[segments_2] [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@15973bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@bea0ae; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_3,generation=3,filenames=[segments_3] [junit4:junit4] 2> 19534 T401 C26 P47486 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3] [junit4:junit4] 2> 19535 T401 C26 P47486 oasu.DirectUpdateHandler2.commit end_commit_flush [junit4:junit4] 2> 19535 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 19535 T401 C26 P47486 /update {waitSearcher=true&softCommit=false&version=2&wt=javabin&commit_end_point=true&commit=true&openSearcher=false} {commit=} 0 2 [junit4:junit4] 2> 19536 T460 C27 P55443 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4:junit4] 2> 19536 T460 C27 P55443 oash.SnapPuller. No value set for 'pollInterval'. Timer Task not started. [junit4:junit4] 2> 19539 T398 C26 P47486 REQ /replication {version=2&qt=/replication&command=indexversion&wt=javabin} status=0 QTime=0 [junit4:junit4] 2> 19539 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Master's generation: 3 [junit4:junit4] 2> 19539 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Slave's generation: 1 [junit4:junit4] 2> 19540 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Starting replication process [junit4:junit4] 2> 19542 T404 C26 P47486 REQ /replication {version=2&qt=/replication&command=filelist&wt=javabin&generation=3} status=0 QTime=1 [junit4:junit4] 2> 19542 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1 [junit4:junit4] 2> 19543 T460 C27 P55443 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1363919394076/1363919408887unloadcollection_3n/index.20130322033013613 [junit4:junit4] 2> 19544 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Starting download to BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@7a682e lockFactory=org.apache.lucene.store.NativeFSLockFactory@8b6574) fullCopy=false [junit4:junit4] 2> 19546 T401 C26 P47486 REQ /replication {checksum=true&qt=/replication&command=filecontent&wt=filestream&file=segments_3&generation=3} status=0 QTime=0 [junit4:junit4] 2> 19548 T460 C27 P55443 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs [junit4:junit4] 2> 19549 T460 C27 P55443 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter... [junit4:junit4] 2> 19549 T460 C27 P55443 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=unloadcollection3 [junit4:junit4] 2> 19550 T460 C27 P55443 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2 [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@16b45d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6af33; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_1,generation=1,filenames=[segments_1] [junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@16b45d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@c6af33; maxCacheMB=0.7197265625 maxMergeSizeMB=0.9306640625))),segFN=segments_3,generation=3,filenames=[segments_3] [junit4:junit4] 2> 19551 T460 C27 P55443 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3] [junit4:junit4] 2> 19552 T460 C27 P55443 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used. [junit4:junit4] 2> 19552 T460 C27 P55443 oass.SolrIndexSearcher. Opening Searcher@1cc236d main [junit4:junit4] 2> 19553 T459 oasc.SolrCore.registerSearcher [unloadcollection3] Registered new searcher Searcher@1cc236d main{StandardDirectoryReader(segments_3:1:nrt)} [junit4:junit4] 2> 19553 T460 C27 P55443 oasc.RecoveryStrategy.replay No replay needed. core=unloadcollection3 [junit4:junit4] 2> 19554 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=unloadcollection3 [junit4:junit4] 2> 19554 T460 C27 P55443 oasc.ZkController.publish publishing core=unloadcollection3 state=active [junit4:junit4] 2> 19554 T460 C27 P55443 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4:junit4] 2> 19556 T460 C27 P55443 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=unloadcollection3 [junit4:junit4] 2> 20136 T391 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 20137 T391 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4:junit4] 2> "core":"unloadcollection3", [junit4:junit4] 2> "numShards":"2", [junit4:junit4] 2> "collection":"unloadcollection", [junit4:junit4] 2> "node_name":"127.0.0.1:55443_iw_", [junit4:junit4] 2> "core_node_name":"127.0.0.1:55443_iw__unloadcollection3", [junit4:junit4] 2> "shard":"shard1", [junit4:junit4] 2> "operation":"state", [junit4:junit4] 2> "base_url":"http://127.0.0.1:55443/iw_", [junit4:junit4] 2> "roles":null, [junit4:junit4] 2> "state":"active"} [junit4:junit4] 2> 20141 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 20141 T422 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 20141 T390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 20141 T436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 20141 T408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 20141 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 20141 T395 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4:junit4] 2> 20537 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... [junit4:junit4] 2> 20537 T368 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: unloadcollection [junit4:junit4] 2> 20538 T368 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false [junit4:junit4] 2> C27_STATE=coll:unloadcollection core:unloadcollection3 props:{core=unloadcollection3, collection=unloadcollection, node_name=127.0.0.1:55443_iw_, shard=shard1, base_url=http://127.0.0.1:55443/iw_, state=active} [junit4:junit4] 2> 20577 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[20 (1430173164116115456)]} 0 2 [junit4:junit4] 2> C25_STATE=coll:unloadcollection core:unloadcollection2 props:{core=unloadcollection2, collection=unloadcollection, node_name=127.0.0.1:51183_iw_, shard=shard1, base_url=http://127.0.0.1:51183/iw_, state=active} [junit4:junit4] 2> 20584 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[20 (1430173164116115456)]} 0 2 [junit4:junit4] 2> 20585 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[20 (1430173164116115456)]} 0 24 [junit4:junit4] 2> 20586 T430 C27 P55443 /update {version=2&wt=javabin} {add=[20]} 0 38 [junit4:junit4] 2> 20597 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[21 (1430173164147572736)]} 0 3 [junit4:junit4] 2> 20597 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[21 (1430173164147572736)]} 0 3 [junit4:junit4] 2> 20603 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[21 (1430173164147572736)]} 0 13 [junit4:junit4] 2> 20603 T429 C27 P55443 /update {version=2&wt=javabin} {add=[21]} 0 15 [junit4:junit4] 2> 20613 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[22 (1430173164165398528)]} 0 1 [junit4:junit4] 2> 20615 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[22 (1430173164165398528)]} 0 3 [junit4:junit4] 2> 20616 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[22 (1430173164165398528)]} 0 8 [junit4:junit4] 2> 20616 T431 C27 P55443 /update {version=2&wt=javabin} {add=[22]} 0 10 [junit4:junit4] 2> 20626 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[23 (1430173164179030016)]} 0 2 [junit4:junit4] 2> 20626 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[23 (1430173164179030016)]} 0 2 [junit4:junit4] 2> 20627 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[23 (1430173164179030016)]} 0 6 [junit4:junit4] 2> 20627 T432 C27 P55443 /update {version=2&wt=javabin} {add=[23]} 0 8 [junit4:junit4] 2> 20636 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[24 (1430173164190564352)]} 0 1 [junit4:junit4] 2> 20637 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[24 (1430173164190564352)]} 0 2 [junit4:junit4] 2> 20638 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[24 (1430173164190564352)]} 0 5 [junit4:junit4] 2> 20638 T430 C27 P55443 /update {version=2&wt=javabin} {add=[24]} 0 8 [junit4:junit4] 2> 20648 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[25 (1430173164202098688)]} 0 2 [junit4:junit4] 2> 20648 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[25 (1430173164202098688)]} 0 2 [junit4:junit4] 2> 20649 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[25 (1430173164202098688)]} 0 6 [junit4:junit4] 2> 20649 T429 C27 P55443 /update {version=2&wt=javabin} {add=[25]} 0 8 [junit4:junit4] 2> 20658 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[26 (1430173164213633024)]} 0 1 [junit4:junit4] 2> 20658 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[26 (1430173164213633024)]} 0 1 [junit4:junit4] 2> 20659 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[26 (1430173164213633024)]} 0 5 [junit4:junit4] 2> 20660 T431 C27 P55443 /update {version=2&wt=javabin} {add=[26]} 0 8 [junit4:junit4] 2> 20668 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[27 (1430173164224118784)]} 0 2 [junit4:junit4] 2> 20668 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[27 (1430173164224118784)]} 0 2 [junit4:junit4] 2> 20669 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[27 (1430173164224118784)]} 0 5 [junit4:junit4] 2> 20669 T432 C27 P55443 /update {version=2&wt=javabin} {add=[27]} 0 7 [junit4:junit4] 2> 20678 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[28 (1430173164234604544)]} 0 2 [junit4:junit4] 2> 20678 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[28 (1430173164234604544)]} 0 1 [junit4:junit4] 2> 20678 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[28 (1430173164234604544)]} 0 5 [junit4:junit4] 2> 20679 T430 C27 P55443 /update {version=2&wt=javabin} {add=[28]} 0 7 [junit4:junit4] 2> 20687 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[29 (1430173164244041728)]} 0 1 [junit4:junit4] 2> 20687 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[29 (1430173164244041728)]} 0 1 [junit4:junit4] 2> 20687 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[29 (1430173164244041728)]} 0 4 [junit4:junit4] 2> 20688 T429 C27 P55443 /update {version=2&wt=javabin} {add=[29]} 0 7 [junit4:junit4] 2> 20695 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[30 (1430173164253478912)]} 0 1 [junit4:junit4] 2> 20695 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[30 (1430173164253478912)]} 0 1 [junit4:junit4] 2> 20696 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[30 (1430173164253478912)]} 0 5 [junit4:junit4] 2> 20696 T431 C27 P55443 /update {version=2&wt=javabin} {add=[30]} 0 6 [junit4:junit4] 2> 20704 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[31 (1430173164261867520)]} 0 1 [junit4:junit4] 2> 20704 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[31 (1430173164261867520)]} 0 1 [junit4:junit4] 2> 20705 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[31 (1430173164261867520)]} 0 5 [junit4:junit4] 2> 20705 T432 C27 P55443 /update {version=2&wt=javabin} {add=[31]} 0 7 [junit4:junit4] 2> 20714 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[32 (1430173164272353280)]} 0 1 [junit4:junit4] 2> 20714 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[32 (1430173164272353280)]} 0 1 [junit4:junit4] 2> 20714 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[32 (1430173164272353280)]} 0 5 [junit4:junit4] 2> 20715 T430 C27 P55443 /update {version=2&wt=javabin} {add=[32]} 0 8 [junit4:junit4] 2> 20723 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[33 (1430173164281790464)]} 0 1 [junit4:junit4] 2> 20723 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[33 (1430173164281790464)]} 0 1 [junit4:junit4] 2> 20724 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[33 (1430173164281790464)]} 0 5 [junit4:junit4] 2> 20725 T429 C27 P55443 /update {version=2&wt=javabin} {add=[33]} 0 8 [junit4:junit4] 2> 20733 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[34 (1430173164292276224)]} 0 1 [junit4:junit4] 2> 20733 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[34 (1430173164292276224)]} 0 1 [junit4:junit4] 2> 20734 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[34 (1430173164292276224)]} 0 5 [junit4:junit4] 2> 20734 T431 C27 P55443 /update {version=2&wt=javabin} {add=[34]} 0 7 [junit4:junit4] 2> 20743 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[35 (1430173164302761984)]} 0 1 [junit4:junit4] 2> 20743 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[35 (1430173164302761984)]} 0 1 [junit4:junit4] 2> 20744 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[35 (1430173164302761984)]} 0 5 [junit4:junit4] 2> 20744 T432 C27 P55443 /update {version=2&wt=javabin} {add=[35]} 0 7 [junit4:junit4] 2> 20752 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[36 (1430173164313247744)]} 0 1 [junit4:junit4] 2> 20752 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[36 (1430173164313247744)]} 0 1 [junit4:junit4] 2> 20753 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[36 (1430173164313247744)]} 0 4 [junit4:junit4] 2> 20754 T430 C27 P55443 /update {version=2&wt=javabin} {add=[36]} 0 7 [junit4:junit4] 2> 20762 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[37 (1430173164322684928)]} 0 1 [junit4:junit4] 2> 20762 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[37 (1430173164322684928)]} 0 1 [junit4:junit4] 2> 20763 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[37 (1430173164322684928)]} 0 5 [junit4:junit4] 2> 20763 T429 C27 P55443 /update {version=2&wt=javabin} {add=[37]} 0 7 [junit4:junit4] 2> 20771 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[38 (1430173164332122112)]} 0 1 [junit4:junit4] 2> 20771 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[38 (1430173164332122112)]} 0 1 [junit4:junit4] 2> 20771 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[38 (1430173164332122112)]} 0 4 [junit4:junit4] 2> 20772 T431 C27 P55443 /update {version=2&wt=javabin} {add=[38]} 0 7 [junit4:junit4] 2> 20780 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[39 (1430173164341559296)]} 0 1 [junit4:junit4] 2> 20780 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[39 (1430173164341559296)]} 0 1 [junit4:junit4] 2> 20781 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[39 (1430173164341559296)]} 0 5 [junit4:junit4] 2> 20781 T432 C27 P55443 /update {version=2&wt=javabin} {add=[39]} 0 7 [junit4:junit4] 2> 20788 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[40 (1430173164350996480)]} 0 0 [junit4:junit4] 2> 20789 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[40 (1430173164350996480)]} 0 1 [junit4:junit4] 2> 20790 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[40 (1430173164350996480)]} 0 5 [junit4:junit4] 2> 20790 T430 C27 P55443 /update {version=2&wt=javabin} {add=[40]} 0 7 [junit4:junit4] 2> 20798 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[41 (1430173164360433664)]} 0 2 [junit4:junit4] 2> 20798 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[41 (1430173164360433664)]} 0 2 [junit4:junit4] 2> 20799 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[41 (1430173164360433664)]} 0 5 [junit4:junit4] 2> 20799 T429 C27 P55443 /update {version=2&wt=javabin} {add=[41]} 0 7 [junit4:junit4] 2> 20807 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[42 (1430173164369870848)]} 0 1 [junit4:junit4] 2> 20807 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[42 (1430173164369870848)]} 0 1 [junit4:junit4] 2> 20808 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[42 (1430173164369870848)]} 0 5 [junit4:junit4] 2> 20808 T431 C27 P55443 /update {version=2&wt=javabin} {add=[42]} 0 7 [junit4:junit4] 2> 20816 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[43 (1430173164379308032)]} 0 1 [junit4:junit4] 2> 20816 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[43 (1430173164379308032)]} 0 1 [junit4:junit4] 2> 20817 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[43 (1430173164379308032)]} 0 5 [junit4:junit4] 2> 20818 T432 C27 P55443 /update {version=2&wt=javabin} {add=[43]} 0 6 [junit4:junit4] 2> 20825 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[44 (1430173164389793792)]} 0 1 [junit4:junit4] 2> 20826 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[44 (1430173164389793792)]} 0 2 [junit4:junit4] 2> 20826 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[44 (1430173164389793792)]} 0 5 [junit4:junit4] 2> 20827 T430 C27 P55443 /update {version=2&wt=javabin} {add=[44]} 0 7 [junit4:junit4] 2> 20835 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[45 (1430173164399230976)]} 0 1 [junit4:junit4] 2> 20835 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[45 (1430173164399230976)]} 0 1 [junit4:junit4] 2> 20836 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[45 (1430173164399230976)]} 0 5 [junit4:junit4] 2> 20836 T429 C27 P55443 /update {version=2&wt=javabin} {add=[45]} 0 7 [junit4:junit4] 2> 20848 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[46 (1430173164409716736)]} 0 1 [junit4:junit4] 2> 20848 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[46 (1430173164409716736)]} 0 1 [junit4:junit4] 2> 20849 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[46 (1430173164409716736)]} 0 8 [junit4:junit4] 2> 20849 T431 C27 P55443 /update {version=2&wt=javabin} {add=[46]} 0 10 [junit4:junit4] 2> 20858 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[47 (1430173164423348224)]} 0 1 [junit4:junit4] 2> 20858 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[47 (1430173164423348224)]} 0 1 [junit4:junit4] 2> 20859 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[47 (1430173164423348224)]} 0 5 [junit4:junit4] 2> 20859 T432 C27 P55443 /update {version=2&wt=javabin} {add=[47]} 0 7 [junit4:junit4] 2> 20867 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[48 (1430173164432785408)]} 0 1 [junit4:junit4] 2> 20867 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[48 (1430173164432785408)]} 0 1 [junit4:junit4] 2> 20868 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[48 (1430173164432785408)]} 0 5 [junit4:junit4] 2> 20869 T430 C27 P55443 /update {version=2&wt=javabin} {add=[48]} 0 8 [junit4:junit4] 2> 20877 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[49 (1430173164443271168)]} 0 1 [junit4:junit4] 2> 20877 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[49 (1430173164443271168)]} 0 1 [junit4:junit4] 2> 20877 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[49 (1430173164443271168)]} 0 4 [junit4:junit4] 2> 20878 T429 C27 P55443 /update {version=2&wt=javabin} {add=[49]} 0 7 [junit4:junit4] 2> 20885 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[50 (1430173164452708352)]} 0 1 [junit4:junit4] 2> 20885 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[50 (1430173164452708352)]} 0 1 [junit4:junit4] 2> 20886 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[50 (1430173164452708352)]} 0 4 [junit4:junit4] 2> 20886 T431 C27 P55443 /update {version=2&wt=javabin} {add=[50]} 0 6 [junit4:junit4] 2> 20894 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[51 (1430173164461096960)]} 0 1 [junit4:junit4] 2> 20894 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[51 (1430173164461096960)]} 0 1 [junit4:junit4] 2> 20895 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[51 (1430173164461096960)]} 0 5 [junit4:junit4] 2> 20896 T432 C27 P55443 /update {version=2&wt=javabin} {add=[51]} 0 8 [junit4:junit4] 2> 20903 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[52 (1430173164471582720)]} 0 0 [junit4:junit4] 2> 20903 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[52 (1430173164471582720)]} 0 0 [junit4:junit4] 2> 20904 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[52 (1430173164471582720)]} 0 4 [junit4:junit4] 2> 20905 T430 C27 P55443 /update {version=2&wt=javabin} {add=[52]} 0 7 [junit4:junit4] 2> 20912 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[53 (1430173164481019904)]} 0 1 [junit4:junit4] 2> 20913 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[53 (1430173164481019904)]} 0 1 [junit4:junit4] 2> 20913 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[53 (1430173164481019904)]} 0 4 [junit4:junit4] 2> 20914 T429 C27 P55443 /update {version=2&wt=javabin} {add=[53]} 0 7 [junit4:junit4] 2> 20921 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[54 (1430173164490457088)]} 0 1 [junit4:junit4] 2> 20921 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[54 (1430173164490457088)]} 0 1 [junit4:junit4] 2> 20922 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[54 (1430173164490457088)]} 0 4 [junit4:junit4] 2> 20922 T431 C27 P55443 /update {version=2&wt=javabin} {add=[54]} 0 6 [junit4:junit4] 2> 20930 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[55 (1430173164499894272)]} 0 1 [junit4:junit4] 2> 20930 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[55 (1430173164499894272)]} 0 1 [junit4:junit4] 2> 20931 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[55 (1430173164499894272)]} 0 5 [junit4:junit4] 2> 20931 T432 C27 P55443 /update {version=2&wt=javabin} {add=[55]} 0 7 [junit4:junit4] 2> 20939 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[56 (1430173164509331456)]} 0 1 [junit4:junit4] 2> 20939 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[56 (1430173164509331456)]} 0 1 [junit4:junit4] 2> 20940 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[56 (1430173164509331456)]} 0 5 [junit4:junit4] 2> 20941 T430 C27 P55443 /update {version=2&wt=javabin} {add=[56]} 0 7 [junit4:junit4] 2> 20949 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[57 (1430173164518768640)]} 0 1 [junit4:junit4] 2> 20949 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[57 (1430173164518768640)]} 0 1 [junit4:junit4] 2> 20950 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[57 (1430173164518768640)]} 0 5 [junit4:junit4] 2> 20950 T432 C27 P55443 /update {version=2&wt=javabin} {add=[57]} 0 7 [junit4:junit4] 2> 20958 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[58 (1430173164529254400)]} 0 1 [junit4:junit4] 2> 20958 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[58 (1430173164529254400)]} 0 1 [junit4:junit4] 2> 20959 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[58 (1430173164529254400)]} 0 5 [junit4:junit4] 2> 20959 T430 C27 P55443 /update {version=2&wt=javabin} {add=[58]} 0 6 [junit4:junit4] 2> 20967 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[59 (1430173164537643008)]} 0 1 [junit4:junit4] 2> 20967 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[59 (1430173164537643008)]} 0 1 [junit4:junit4] 2> 20967 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[59 (1430173164537643008)]} 0 4 [junit4:junit4] 2> 20968 T431 C27 P55443 /update {version=2&wt=javabin} {add=[59]} 0 7 [junit4:junit4] 2> 20985 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[60 (1430173164555468800)]} 0 1 [junit4:junit4] 2> 20986 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[60 (1430173164555468800)]} 0 1 [junit4:junit4] 2> 20987 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[60 (1430173164555468800)]} 0 7 [junit4:junit4] 2> 20988 T429 C27 P55443 /update {version=2&wt=javabin} {add=[60]} 0 10 [junit4:junit4] 2> 20995 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[61 (1430173164568051712)]} 0 1 [junit4:junit4] 2> 20995 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[61 (1430173164568051712)]} 0 1 [junit4:junit4] 2> 20996 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[61 (1430173164568051712)]} 0 4 [junit4:junit4] 2> 20996 T432 C27 P55443 /update {version=2&wt=javabin} {add=[61]} 0 6 [junit4:junit4] 2> 21004 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[62 (1430173164577488896)]} 0 1 [junit4:junit4] 2> 21004 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[62 (1430173164577488896)]} 0 1 [junit4:junit4] 2> 21005 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[62 (1430173164577488896)]} 0 5 [junit4:junit4] 2> 21005 T430 C27 P55443 /update {version=2&wt=javabin} {add=[62]} 0 6 [junit4:junit4] 2> 21013 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[63 (1430173164585877504)]} 0 1 [junit4:junit4] 2> 21013 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[63 (1430173164585877504)]} 0 1 [junit4:junit4] 2> 21014 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[63 (1430173164585877504)]} 0 5 [junit4:junit4] 2> 21014 T431 C27 P55443 /update {version=2&wt=javabin} {add=[63]} 0 7 [junit4:junit4] 2> 21021 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[64 (1430173164595314688)]} 0 1 [junit4:junit4] 2> 21021 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[64 (1430173164595314688)]} 0 1 [junit4:junit4] 2> 21022 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[64 (1430173164595314688)]} 0 4 [junit4:junit4] 2> 21023 T429 C27 P55443 /update {version=2&wt=javabin} {add=[64]} 0 7 [junit4:junit4] 2> 21031 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[65 (1430173164604751872)]} 0 1 [junit4:junit4] 2> 21031 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[65 (1430173164604751872)]} 0 1 [junit4:junit4] 2> 21032 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[65 (1430173164604751872)]} 0 5 [junit4:junit4] 2> 21032 T432 C27 P55443 /update {version=2&wt=javabin} {add=[65]} 0 7 [junit4:junit4] 2> 21040 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[66 (1430173164614189056)]} 0 1 [junit4:junit4] 2> 21040 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[66 (1430173164614189056)]} 0 1 [junit4:junit4] 2> 21041 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[66 (1430173164614189056)]} 0 5 [junit4:junit4] 2> 21041 T430 C27 P55443 /update {version=2&wt=javabin} {add=[66]} 0 7 [junit4:junit4] 2> 21048 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[67 (1430173164623626240)]} 0 1 [junit4:junit4] 2> 21048 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[67 (1430173164623626240)]} 0 0 [junit4:junit4] 2> 21049 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[67 (1430173164623626240)]} 0 4 [junit4:junit4] 2> 21049 T431 C27 P55443 /update {version=2&wt=javabin} {add=[67]} 0 6 [junit4:junit4] 2> 21057 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[68 (1430173164633063424)]} 0 1 [junit4:junit4] 2> 21057 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[68 (1430173164633063424)]} 0 1 [junit4:junit4] 2> 21058 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[68 (1430173164633063424)]} 0 5 [junit4:junit4] 2> 21058 T429 C27 P55443 /update {version=2&wt=javabin} {add=[68]} 0 7 [junit4:junit4] 2> 21065 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[69 (1430173164641452032)]} 0 0 [junit4:junit4] 2> 21065 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[69 (1430173164641452032)]} 0 0 [junit4:junit4] 2> 21066 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[69 (1430173164641452032)]} 0 4 [junit4:junit4] 2> 21067 T432 C27 P55443 /update {version=2&wt=javabin} {add=[69]} 0 7 [junit4:junit4] 2> 21074 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[70 (1430173164650889216)]} 0 1 [junit4:junit4] 2> 21074 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[70 (1430173164650889216)]} 0 0 [junit4:junit4] 2> 21075 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[70 (1430173164650889216)]} 0 4 [junit4:junit4] 2> 21076 T430 C27 P55443 /update {version=2&wt=javabin} {add=[70]} 0 7 [junit4:junit4] 2> 21083 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[71 (1430173164660326400)]} 0 1 [junit4:junit4] 2> 21083 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[71 (1430173164660326400)]} 0 1 [junit4:junit4] 2> 21083 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[71 (1430173164660326400)]} 0 4 [junit4:junit4] 2> 21084 T431 C27 P55443 /update {version=2&wt=javabin} {add=[71]} 0 6 [junit4:junit4] 2> 21091 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[72 (1430173164668715008)]} 0 1 [junit4:junit4] 2> 21091 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[72 (1430173164668715008)]} 0 1 [junit4:junit4] 2> 21092 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[72 (1430173164668715008)]} 0 4 [junit4:junit4] 2> 21092 T429 C27 P55443 /update {version=2&wt=javabin} {add=[72]} 0 6 [junit4:junit4] 2> 21099 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[73 (1430173164677103616)]} 0 1 [junit4:junit4] 2> 21099 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[73 (1430173164677103616)]} 0 1 [junit4:junit4] 2> 21099 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[73 (1430173164677103616)]} 0 3 [junit4:junit4] 2> 21100 T432 C27 P55443 /update {version=2&wt=javabin} {add=[73]} 0 6 [junit4:junit4] 2> 21107 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[74 (1430173164685492224)]} 0 1 [junit4:junit4] 2> 21107 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[74 (1430173164685492224)]} 0 1 [junit4:junit4] 2> 21108 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[74 (1430173164685492224)]} 0 4 [junit4:junit4] 2> 21108 T430 C27 P55443 /update {version=2&wt=javabin} {add=[74]} 0 6 [junit4:junit4] 2> 21116 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[75 (1430173164693880832)]} 0 1 [junit4:junit4] 2> 21117 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[75 (1430173164693880832)]} 0 1 [junit4:junit4] 2> 21117 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[75 (1430173164693880832)]} 0 5 [junit4:junit4] 2> 21118 T431 C27 P55443 /update {version=2&wt=javabin} {add=[75]} 0 8 [junit4:junit4] 2> 21126 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[76 (1430173164704366592)]} 0 1 [junit4:junit4] 2> 21126 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[76 (1430173164704366592)]} 0 1 [junit4:junit4] 2> 21127 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[76 (1430173164704366592)]} 0 5 [junit4:junit4] 2> 21127 T429 C27 P55443 /update {version=2&wt=javabin} {add=[76]} 0 7 [junit4:junit4] 2> 21135 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[77 (1430173164713803776)]} 0 1 [junit4:junit4] 2> 21135 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[77 (1430173164713803776)]} 0 1 [junit4:junit4] 2> 21136 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[77 (1430173164713803776)]} 0 5 [junit4:junit4] 2> 21136 T432 C27 P55443 /update {version=2&wt=javabin} {add=[77]} 0 7 [junit4:junit4] 2> 21143 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[78 (1430173164724289536)]} 0 0 [junit4:junit4] 2> 21144 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[78 (1430173164724289536)]} 0 1 [junit4:junit4] 2> 21144 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[78 (1430173164724289536)]} 0 4 [junit4:junit4] 2> 21145 T430 C27 P55443 /update {version=2&wt=javabin} {add=[78]} 0 6 [junit4:junit4] 2> 21152 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[79 (1430173164732678144)]} 0 1 [junit4:junit4] 2> 21152 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[79 (1430173164732678144)]} 0 1 [junit4:junit4] 2> 21153 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[79 (1430173164732678144)]} 0 5 [junit4:junit4] 2> 21153 T431 C27 P55443 /update {version=2&wt=javabin} {add=[79]} 0 6 [junit4:junit4] 2> 21160 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[80 (1430173164741066752)]} 0 1 [junit4:junit4] 2> 21160 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[80 (1430173164741066752)]} 0 1 [junit4:junit4] 2> 21161 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[80 (1430173164741066752)]} 0 4 [junit4:junit4] 2> 21161 T429 C27 P55443 /update {version=2&wt=javabin} {add=[80]} 0 6 [junit4:junit4] 2> 21168 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[81 (1430173164749455360)]} 0 1 [junit4:junit4] 2> 21168 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[81 (1430173164749455360)]} 0 0 [junit4:junit4] 2> 21169 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[81 (1430173164749455360)]} 0 4 [junit4:junit4] 2> 21170 T432 C27 P55443 /update {version=2&wt=javabin} {add=[81]} 0 7 [junit4:junit4] 2> 21176 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[82 (1430173164757843968)]} 0 0 [junit4:junit4] 2> 21176 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[82 (1430173164757843968)]} 0 0 [junit4:junit4] 2> 21177 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[82 (1430173164757843968)]} 0 4 [junit4:junit4] 2> 21177 T430 C27 P55443 /update {version=2&wt=javabin} {add=[82]} 0 5 [junit4:junit4] 2> 21184 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[83 (1430173164766232576)]} 0 0 [junit4:junit4] 2> 21185 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[83 (1430173164766232576)]} 0 1 [junit4:junit4] 2> 21185 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[83 (1430173164766232576)]} 0 4 [junit4:junit4] 2> 21186 T431 C27 P55443 /update {version=2&wt=javabin} {add=[83]} 0 7 [junit4:junit4] 2> 21192 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[84 (1430173164775669760)]} 0 0 [junit4:junit4] 2> 21192 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[84 (1430173164775669760)]} 0 0 [junit4:junit4] 2> 21193 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[84 (1430173164775669760)]} 0 4 [junit4:junit4] 2> 21193 T429 C27 P55443 /update {version=2&wt=javabin} {add=[84]} 0 5 [junit4:junit4] 2> 21200 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[85 (1430173164783009792)]} 0 1 [junit4:junit4] 2> 21200 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[85 (1430173164783009792)]} 0 1 [junit4:junit4] 2> 21201 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[85 (1430173164783009792)]} 0 4 [junit4:junit4] 2> 21201 T432 C27 P55443 /update {version=2&wt=javabin} {add=[85]} 0 6 [junit4:junit4] 2> 21208 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[86 (1430173164791398400)]} 0 1 [junit4:junit4] 2> 21208 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[86 (1430173164791398400)]} 0 1 [junit4:junit4] 2> 21209 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[86 (1430173164791398400)]} 0 4 [junit4:junit4] 2> 21209 T430 C27 P55443 /update {version=2&wt=javabin} {add=[86]} 0 6 [junit4:junit4] 2> 21216 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[87 (1430173164799787008)]} 0 1 [junit4:junit4] 2> 21216 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[87 (1430173164799787008)]} 0 1 [junit4:junit4] 2> 21217 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[87 (1430173164799787008)]} 0 4 [junit4:junit4] 2> 21217 T431 C27 P55443 /update {version=2&wt=javabin} {add=[87]} 0 6 [junit4:junit4] 2> 21224 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[88 (1430173164808175616)]} 0 1 [junit4:junit4] 2> 21224 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[88 (1430173164808175616)]} 0 1 [junit4:junit4] 2> 21224 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[88 (1430173164808175616)]} 0 3 [junit4:junit4] 2> 21225 T429 C27 P55443 /update {version=2&wt=javabin} {add=[88]} 0 6 [junit4:junit4] 2> 21232 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[89 (1430173164816564224)]} 0 1 [junit4:junit4] 2> 21232 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[89 (1430173164816564224)]} 0 1 [junit4:junit4] 2> 21233 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[89 (1430173164816564224)]} 0 4 [junit4:junit4] 2> 21233 T432 C27 P55443 /update {version=2&wt=javabin} {add=[89]} 0 6 [junit4:junit4] 2> 21240 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[90 (1430173164826001408)]} 0 0 [junit4:junit4] 2> 21241 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[90 (1430173164826001408)]} 0 1 [junit4:junit4] 2> 21241 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[90 (1430173164826001408)]} 0 4 [junit4:junit4] 2> 21242 T430 C27 P55443 /update {version=2&wt=javabin} {add=[90]} 0 6 [junit4:junit4] 2> 21249 T415 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[91 (1430173164834390016)]} 0 0 [junit4:junit4] 2> 21249 T432 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[91 (1430173164834390016)]} 0 1 [junit4:junit4] 2> 21249 T401 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[91 (1430173164834390016)]} 0 4 [junit4:junit4] 2> 21250 T431 C27 P55443 /update {version=2&wt=javabin} {add=[91]} 0 6 [junit4:junit4] 2> 21256 T430 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[92 (1430173164842778624)]} 0 0 [junit4:junit4] 2> 21256 T416 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[92 (1430173164842778624)]} 0 0 [junit4:junit4] 2> 21257 T398 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[92 (1430173164842778624)]} 0 3 [junit4:junit4] 2> 21258 T429 C27 P55443 /update {version=2&wt=javabin} {add=[92]} 0 6 [junit4:junit4] 2> 21265 T418 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[93 (1430173164851167232)]} 0 1 [junit4:junit4] 2> 21268 T431 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[93 (1430173164851167232)]} 0 4 [junit4:junit4] 2> 21268 T404 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[93 (1430173164851167232)]} 0 7 [junit4:junit4] 2> 21269 T432 C27 P55443 /update {version=2&wt=javabin} {add=[93]} 0 9 [junit4:junit4] 2> 21276 T429 C27 P55443 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[94 (1430173164862701568)]} 0 1 [junit4:junit4] 2> 21276 T417 C25 P51183 /update {distrib.from=http://127.0.0.1:47486/iw_/unloadcollection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[94 (1430173164862701568)]} 0 1 [junit4:junit4] 2> 21277 T402 C26 P47486 /update {distrib.from=http://127.0.0.1:55443/iw_/unloadcollection3/&version=2&wt=javabin&update.distrib=TOLEADER} {add=[94 (1430173164862701568)]} 0 4 [junit4:junit4] 2> [...truncated too long message...] afe stop: [junit4:junit4] 1> safe stop: [junit4:junit4] 1> safe stop: [junit4:junit4] 2> 34488 T747 oas.SolrTestCaseJ4.deleteCore ###deleteCore [junit4:junit4] 2> 34491 T746 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s). [junit4:junit4] 2> 34588 T750 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop! [junit4:junit4] 2> 35703 T831 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK [junit4:junit4] 2> 35703 T831 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1) [junit4:junit4] 2> 35703 T831 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK [junit4:junit4] 2> 35703 T831 oaz.ClientCnxn$EventThread.run EventThread shut down [junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ga, timezone=America/Belem [junit4:junit4] 2> NOTE: Linux 3.2.0-39-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=1,free=94319912,total=282460160 [junit4:junit4] 2> NOTE: All tests run in this JVM: [TestSweetSpotSimilarityFactory, SynonymTokenizerTest, SearchHandlerTest, ZkControllerTest, TestDocSet, DistributedQueryElevationComponentTest, AlternateDirectoryTest, FileBasedSpellCheckerTest, TestPluginEnable, SuggesterTest, QueryEqualityTest, DistributedTermsComponentTest, TestLFUCache, MoreLikeThisHandlerTest, TestCoreContainer, TestCopyFieldCollectionResource, TestFieldTypeResource, TestLRUCache, TestSolrCoreProperties, TestBadConfig, BasicFunctionalityTest, RAMDirectoryFactoryTest, FieldMutatingUpdateProcessorTest, TestXIncludeConfig, TestQueryUtils, LoggingHandlerTest, UnloadDistributedZkTest, TermVectorComponentTest, QueryParsingTest, TestCharFilters, ResourceLoaderTest, AutoCommitTest, TestRandomDVFaceting, CSVRequestHandlerTest, RegexBoostProcessorTest, TestIBSimilarityFactory, PreAnalyzedFieldTest, CoreAdminHandlerTest, DocValuesTest, OverseerCollectionProcessorTest, RecoveryZkTest] [junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RecoveryZkTest -Dtests.seed=8A99804DB37337B1 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ga -Dtests.timezone=America/Belem -Dtests.file.encoding=ISO-8859-1 [junit4:junit4] ERROR 0.00s J0 | RecoveryZkTest (suite) <<< [junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=806, name=coreLoadExecutor-457-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below. [junit4:junit4] > at java.lang.Thread.getStackTrace(Thread.java:1569) [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150) [junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:524) [junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:991) [junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:883) [junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:875) [junit4:junit4] > at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:37) [junit4:junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:327) [junit4:junit4] > at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:244) [junit4:junit4] > at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:458) [junit4:junit4] > at org.apache.solr.core.SolrCore.(SolrCore.java:746) [junit4:junit4] > at org.apache.solr.core.SolrCore.(SolrCore.java:619) [junit4:junit4] > at org.apache.solr.core.CoreContainer.createFromZk(CoreContainer.java:872) [junit4:junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:955) [junit4:junit4] > at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:579) [junit4:junit4] > at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:574) [junit4:junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [junit4:junit4] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [junit4:junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [junit4:junit4] > at java.lang.Thread.run(Thread.java:722) [junit4:junit4] > Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.RAMDirectory@3e2c64 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1588dba) [junit4:junit4] > at org.junit.Assert.fail(Assert.java:93) [junit4:junit4] > at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47) [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602) [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599) [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167) [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615) [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43) [junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) [junit4:junit4] > ... 1 more [junit4:junit4] > Throwable #2: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=808, name=RecoveryThread, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below. [junit4:junit4] > at java.lang.Thread.getStackTrace(Thread.java:1569) [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150) [junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:524) [junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:991) [junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:883) [junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:875) [junit4:junit4] > at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:37) [junit4:junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:327) [junit4:junit4] > at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:383) [junit4:junit4] > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:281) [junit4:junit4] > at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:153) [junit4:junit4] > at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409) [junit4:junit4] > at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:223) [junit4:junit4] > Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ca4702 lockFactory=org.apache.lucene.store.NativeFSLockFactory@41fbf7) [junit4:junit4] > at org.junit.Assert.fail(Assert.java:93) [junit4:junit4] > at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47) [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602) [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599) [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167) [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615) [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43) [junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) [junit4:junit4] > at java.lang.Thread.run(Thread.java:722) [junit4:junit4] Completed on J0 in 35.79s, 1 test, 2 errors <<< FAILURES! [...truncated 519 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:375: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:355: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:183: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:449: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1210: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:874: There were test failures: 272 suites, 1145 tests, 2 suite-level errors, 1 failure, 13 ignored (7 assumptions) Total time: 37 minutes 37 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Description set: Java: 32bit/jdk1.8.0-ea-b79 -client -XX:+UseParallelGC Email was triggered for: Failure Sending email for trigger: Failure ------=_Part_100_306175094.1363920556112 Content-Type: text/plain; charset=us-ascii --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org ------=_Part_100_306175094.1363920556112--