Return-Path: X-Original-To: apmail-lucene-dev-archive@www.apache.org Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1A9F5FB4C for ; Wed, 17 Apr 2013 08:23:22 +0000 (UTC) Received: (qmail 62931 invoked by uid 500); 17 Apr 2013 08:23:20 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 62563 invoked by uid 500); 17 Apr 2013 08:23:19 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 62532 invoked by uid 99); 17 Apr 2013 08:23:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 17 Apr 2013 08:23:18 +0000 X-ASF-Spam-Status: No, hits=0.0 required=5.0 tests=NORMAL_HTTP_TO_IP,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jenkins@thetaphi.de designates 188.138.97.18 as permitted sender) Received: from [188.138.97.18] (HELO mail.sd-datasolutions.de) (188.138.97.18) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 17 Apr 2013 08:23:10 +0000 Received: from serv1 (localhost.localdomain [127.0.0.1]) by mail.sd-datasolutions.de (Postfix) with ESMTP id 7C3AE14AA044 for ; Wed, 17 Apr 2013 08:22:47 +0000 (UTC) Date: Wed, 17 Apr 2013 08:22:31 +0000 (UTC) From: Policeman Jenkins Server To: dev@lucene.apache.org Message-ID: <878057649.15.1366186967380.JavaMail.jenkins@serv1.sd-datasolutions.de> In-Reply-To: <248731556.13.1366182029849.JavaMail.jenkins@serv1.sd-datasolutions.de> References: <248731556.13.1366182029849.JavaMail.jenkins@serv1.sd-datasolutions.de> Subject: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_17) - Build # 5154 - Still Failing! MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_14_1011344083.1366186951786" X-Jenkins-Job: Lucene-Solr-4.x-Linux X-Jenkins-Result: FAILURE X-Virus-Checked: Checked by ClamAV on apache.org ------=_Part_14_1011344083.1366186951786 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/5154/ Java: 32bit/jdk1.7.0_17 -server -XX:+UseSerialGC 1 tests failed. REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDist= ribSearch Error Message: There are still nodes recoverying - waited for 230 seconds Stack Trace: java.lang.AssertionError: There are still nodes recoverying - waited for 23= 0 seconds =09at __randomizedtesting.SeedInfo.seed([DEE6CB3BD5AFCE84:5F004523A2F0AEB8]= :0) =09at org.junit.Assert.fail(Assert.java:93) =09at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFi= nish(AbstractDistribZkTestBase.java:173) =09at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFi= nish(AbstractDistribZkTestBase.java:131) =09at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFi= nish(AbstractDistribZkTestBase.java:126) =09at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollections= API(CollectionsAPIDistributedZkTest.java:509) =09at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(Collecti= onsAPIDistributedZkTest.java:145) =09at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseD= istributedSearchTestCase.java:806) =09at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) =09at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.= java:57) =09at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces= sorImpl.java:43) =09at java.lang.reflect.Method.invoke(Method.java:601) =09at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomized= Runner.java:1559) =09at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(Random= izedRunner.java:79) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Random= izedRunner.java:737) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Random= izedRunner.java:773) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Random= izedRunner.java:787) =09at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$= 1.evaluate(SystemPropertiesRestoreRule.java:53) =09at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRu= leSetupTeardownChained.java:50) =09at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFi= eldCacheSanity.java:51) =09at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBef= oreAfterRule.java:46) =09at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRul= e$1.evaluate(SystemPropertiesInvariantRule.java:55) =09at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleT= hreadAndTestName.java:49) =09at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(Test= RuleIgnoreAfterMaxFailures.java:70) =09at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFai= lure.java:48) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.= run(ThreadLeakControl.java:358) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTa= sk(ThreadLeakControl.java:782) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(Threa= dLeakControl.java:442) =09at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Ran= domizedRunner.java:746) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(Random= izedRunner.java:648) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(Random= izedRunner.java:682) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Random= izedRunner.java:693) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$= 1.evaluate(SystemPropertiesRestoreRule.java:53) =09at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBef= oreAfterRule.java:46) =09at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStor= eClassName.java:42) =09at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRul= e$1.evaluate(SystemPropertiesInvariantRule.java:55) =09at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMeth= odsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) =09at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMeth= odsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRule= AssertionsRequired.java:43) =09at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFai= lure.java:48) =09at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(Test= RuleIgnoreAfterMaxFailures.java:70) =09at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIg= noreTestSuites.java:55) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.= run(ThreadLeakControl.java:358) =09at java.lang.Thread.run(Thread.java:722) Build Log: [...truncated 9430 lines...] [junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTes= t [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.259; org.apache.solr.BaseD= istributedSearchTestCase; Setting hostContext system property: /_r/eq [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.262; org.apache.solr.SolrT= estCaseJ4; ###Starting testDistribSearch [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.263; org.apache.solr.cloud= .ZkTestServer; STARTING ZK TEST SERVER [junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.264; org.apache.solr.cloud= .ZkTestServer$ZKServerMain; Starting server [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.364; org.apache.solr.cloud= .ZkTestServer; start zk server on port:40169 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.365; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.433; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@d74ec7 name:ZooKeeperConnection Watcher:127.0.0.1:40169 got event Wa= tchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.433; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.434; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /solr [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.438; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.439; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@1b1d818 name:ZooKeeperConnection Watcher:127.0.0.1:40169/solr got ev= ent WatchedEvent state:SyncConnected type:None path:null path:null type:Non= e [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.439; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.439; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/collection1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.442; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/collection1/shards [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.444; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/control_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.446; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/control_collection/shards [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.449; org.apache.solr.cloud= .AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/s= olr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to= /configs/conf1/solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.450; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /configs/conf1/solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.455; org.apache.solr.cloud= .AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/s= olr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs= /conf1/schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.456; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /configs/conf1/schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.460; org.apache.solr.cloud= .AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/s= olr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /conf= igs/conf1/stopwords.txt [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.461; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /configs/conf1/stopwords.txt [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.464; org.apache.solr.cloud= .AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/s= olr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /conf= igs/conf1/protwords.txt [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.464; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /configs/conf1/protwords.txt [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.466; org.apache.solr.cloud= .AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/s= olr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /confi= gs/conf1/currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.467; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /configs/conf1/currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.469; org.apache.solr.cloud= .AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/s= olr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.js= on to /configs/conf1/open-exchange-rates.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.470; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /configs/conf1/open-exchange-rates.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.472; org.apache.solr.cloud= .AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/s= olr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accen= t.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.472; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.474; org.apache.solr.cloud= .AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/s= olr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /c= onfigs/conf1/old_synonyms.txt [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.475; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /configs/conf1/old_synonyms.txt [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.477; org.apache.solr.cloud= .AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/s= olr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /confi= gs/conf1/synonyms.txt [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.477; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /configs/conf1/synonyms.txt [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.557; org.eclipse.jetty.ser= ver.Server; jetty-8.1.8.v20121106 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.576; org.eclipse.jetty.ser= ver.AbstractConnector; Started SelectChannelConnector@127.0.0.1:58902 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.577; org.apache.solr.servl= et.SolrDispatchFilter; SolrDispatchFilter.init() [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.577; org.apache.solr.core.= SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx) [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.577; org.apache.solr.core.= SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/= workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.s= olr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366186241479 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.578; org.apache.solr.core.= CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366186241479/solr.xm= l [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.578; org.apache.solr.core.= CoreContainer; New CoreContainer 21005545 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.578; org.apache.solr.core.= CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/wor= kspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr= .cloud.CollectionsAPIDistributedZkTest-controljetty-1366186241479/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.579; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366186241479/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.656; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.656; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting urlScheme to: http:// [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.657; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting connTimeout to: 15000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.657; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.657; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting corePoolSize to: 0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.658; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 214748364= 7 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.658; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.658; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.658; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.659; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= sPerHost=3D20&maxConnections=3D10000&socketTimeout=3D120000&connTimeout=3D1= 5000&retry=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.662; org.apache.solr.core.= CoreContainer; Registering Log Listener [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.662; org.apache.solr.core.= CoreContainer; Zookeeper client=3D127.0.0.1:40169/solr [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.663; org.apache.solr.cloud= .ZkController; zkHost includes chroot [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.663; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.665; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@4849da name:ZooKeeperConnection Watcher:127.0.0.1:40169 got event Wa= tchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.665; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.667; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D500&maxConnectionsPerHost=3D16&socketTimeout=3D120000&connTimeout=3D150= 00 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.669; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.671; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@4bac5d name:ZooKeeperConnection Watcher:127.0.0.1:40169/solr got eve= nt WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.671; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.673; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /overseer/queue [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.676; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /overseer/collection-queue-work [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.678; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /live_nodes [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.679; org.apache.solr.cloud= .ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:589= 02__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.680; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:58902__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.682; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /overseer_elect/election [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.705; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /overseer_elect/leader [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.707; org.apache.solr.cloud= .Overseer; Overseer (id=3D89534381507608579-127.0.0.1:58902__r%2Feq-n_00000= 00000) starting [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.709; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /overseer/queue-work [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.711; org.apache.solr.cloud= .OverseerCollectionProcessor; Process current queue of collection creations [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.712; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /clusterstate.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.713; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /aliases.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.715; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cluster state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.717; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Starting to work on the main queue [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.718; org.apache.solr.servl= et.SolrDispatchFilter; user.dir=3D/mnt/ssd/jenkins/workspace/Lucene-Solr-4.= x-Linux/solr/build/solr-core/test/J1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.718; org.apache.solr.servl= et.SolrDispatchFilter; SolrDispatchFilter.init() done [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.719; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.721; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config: [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.722; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.723; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@c0cff1 name:ZooKeeperConnection Watcher:127.0.0.1:40169/solr got eve= nt WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.723; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.724; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cluster state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.726; org.apache.solr.cloud= .ChaosMonkey; monkey: init - expire sessions:true cause connection loss:tru= e [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.796; org.eclipse.jetty.ser= ver.Server; jetty-8.1.8.v20121106 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.797; org.eclipse.jetty.ser= ver.AbstractConnector; Started SelectChannelConnector@127.0.0.1:60700 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.798; org.apache.solr.servl= et.SolrDispatchFilter; SolrDispatchFilter.init() [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.798; org.apache.solr.core.= SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx) [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.799; org.apache.solr.core.= SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/= workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.s= olr.cloud.CollectionsAPIDistributedZkTest-jetty1-1366186241726 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.799; org.apache.solr.core.= CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty1-1366186241726/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.799; org.apache.solr.core.= CoreContainer; New CoreContainer 13583289 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.800; org.apache.solr.core.= CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/wor= kspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr= .cloud.CollectionsAPIDistributedZkTest-jetty1-1366186241726/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.800; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1366186241726/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.872; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.872; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting urlScheme to: http:// [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.873; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting connTimeout to: 15000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.873; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.873; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting corePoolSize to: 0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.874; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 214748364= 7 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.874; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.874; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.875; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.875; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= sPerHost=3D20&maxConnections=3D10000&socketTimeout=3D120000&connTimeout=3D1= 5000&retry=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.878; org.apache.solr.core.= CoreContainer; Registering Log Listener [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.878; org.apache.solr.core.= CoreContainer; Zookeeper client=3D127.0.0.1:40169/solr [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.879; org.apache.solr.cloud= .ZkController; zkHost includes chroot [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.880; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.881; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@2c2ad7 name:ZooKeeperConnection Watcher:127.0.0.1:40169 got event Wa= tchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.881; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.883; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D500&maxConnectionsPerHost=3D16&socketTimeout=3D120000&connTimeout=3D150= 00 [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.886; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.887; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@f76ff5 name:ZooKeeperConnection Watcher:127.0.0.1:40169/solr got eve= nt WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.887; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:41.891; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cluster state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.894; org.apache.solr.cloud= .ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:607= 00__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.895; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:60700__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.898; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (2) [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.898; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (2) [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.898; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (2) [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.901; org.apache.solr.servl= et.SolrDispatchFilter; user.dir=3D/mnt/ssd/jenkins/workspace/Lucene-Solr-4.= x-Linux/solr/build/solr-core/test/J1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.902; org.apache.solr.servl= et.SolrDispatchFilter; SolrDispatchFilter.init() done [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.902; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.970; org.eclipse.jetty.ser= ver.Server; jetty-8.1.8.v20121106 [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.971; org.eclipse.jetty.ser= ver.AbstractConnector; Started SelectChannelConnector@127.0.0.1:51825 [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.972; org.apache.solr.servl= et.SolrDispatchFilter; SolrDispatchFilter.init() [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.973; org.apache.solr.core.= SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx) [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.973; org.apache.solr.core.= SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/= workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.s= olr.cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242905 [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.973; org.apache.solr.core.= CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242905/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.974; org.apache.solr.core.= CoreContainer; New CoreContainer 27269617 [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.974; org.apache.solr.core.= CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/wor= kspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr= .cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242905/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:42.974; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242905/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.050; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.050; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting urlScheme to: http:// [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.051; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting connTimeout to: 15000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.051; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.052; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting corePoolSize to: 0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.052; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 214748364= 7 [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.053; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5 [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.053; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.054; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.054; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= sPerHost=3D20&maxConnections=3D10000&socketTimeout=3D120000&connTimeout=3D1= 5000&retry=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.059; org.apache.solr.core.= CoreContainer; Registering Log Listener [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.059; org.apache.solr.core.= CoreContainer; Zookeeper client=3D127.0.0.1:40169/solr [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.060; org.apache.solr.cloud= .ZkController; zkHost includes chroot [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.060; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.062; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@1e99209 name:ZooKeeperConnection Watcher:127.0.0.1:40169 got event W= atchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.062; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.064; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D500&maxConnectionsPerHost=3D16&socketTimeout=3D120000&connTimeout=3D150= 00 [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.067; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.068; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@d67076 name:ZooKeeperConnection Watcher:127.0.0.1:40169/solr got eve= nt WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.068; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:43.070; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cluster state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.073; org.apache.solr.cloud= .ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:518= 25__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.074; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:51825__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.076; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (3) [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.076; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (3) [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.077; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (3) [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.077; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (3) [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.080; org.apache.solr.servl= et.SolrDispatchFilter; user.dir=3D/mnt/ssd/jenkins/workspace/Lucene-Solr-4.= x-Linux/solr/build/solr-core/test/J1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.080; org.apache.solr.servl= et.SolrDispatchFilter; SolrDispatchFilter.init() done [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.081; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.155; org.eclipse.jetty.ser= ver.Server; jetty-8.1.8.v20121106 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.157; org.eclipse.jetty.ser= ver.AbstractConnector; Started SelectChannelConnector@127.0.0.1:60642 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.158; org.apache.solr.servl= et.SolrDispatchFilter; SolrDispatchFilter.init() [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.158; org.apache.solr.core.= SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx) [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.158; org.apache.solr.core.= SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/= workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.s= olr.cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244083 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.159; org.apache.solr.core.= CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244083/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.159; org.apache.solr.core.= CoreContainer; New CoreContainer 9293568 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.160; org.apache.solr.core.= CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/wor= kspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr= .cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244083/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.160; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244083/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.237; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.238; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting urlScheme to: http:// [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.238; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting connTimeout to: 15000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.238; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.239; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting corePoolSize to: 0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.239; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 214748364= 7 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.240; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.240; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.241; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.241; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= sPerHost=3D20&maxConnections=3D10000&socketTimeout=3D120000&connTimeout=3D1= 5000&retry=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.244; org.apache.solr.core.= CoreContainer; Registering Log Listener [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.245; org.apache.solr.core.= CoreContainer; Zookeeper client=3D127.0.0.1:40169/solr [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.245; org.apache.solr.cloud= .ZkController; zkHost includes chroot [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.246; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.264; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@191df2b name:ZooKeeperConnection Watcher:127.0.0.1:40169 got event W= atchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.264; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.283; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D500&maxConnectionsPerHost=3D16&socketTimeout=3D120000&connTimeout=3D150= 00 [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.289; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.303; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@1ecd9dc name:ZooKeeperConnection Watcher:127.0.0.1:40169/solr got ev= ent WatchedEvent state:SyncConnected type:None path:null path:null type:Non= e [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.303; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:44.345; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cluster state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.349; org.apache.solr.cloud= .ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:606= 42__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.351; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:60642__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.355; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (4) [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.355; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (4) [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.355; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (4) [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.355; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (4) [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.355; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (4) [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.360; org.apache.solr.servl= et.SolrDispatchFilter; user.dir=3D/mnt/ssd/jenkins/workspace/Lucene-Solr-4.= x-Linux/solr/build/solr-core/test/J1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.361; org.apache.solr.servl= et.SolrDispatchFilter; SolrDispatchFilter.init() done [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.361; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.426; org.eclipse.jetty.ser= ver.Server; jetty-8.1.8.v20121106 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.428; org.eclipse.jetty.ser= ver.AbstractConnector; Started SelectChannelConnector@127.0.0.1:49741 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.428; org.apache.solr.servl= et.SolrDispatchFilter; SolrDispatchFilter.init() [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.429; org.apache.solr.core.= SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx) [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.429; org.apache.solr.core.= SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/= workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.s= olr.cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245364 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.429; org.apache.solr.core.= CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245364/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.430; org.apache.solr.core.= CoreContainer; New CoreContainer 3900299 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.430; org.apache.solr.core.= CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/wor= kspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr= .cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245364/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.430; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245364/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.516; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.517; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting urlScheme to: http:// [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.517; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting connTimeout to: 15000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.517; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.518; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting corePoolSize to: 0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.518; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 214748364= 7 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.518; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.519; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.519; org.apache.solr.handl= er.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.520; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= sPerHost=3D20&maxConnections=3D10000&socketTimeout=3D120000&connTimeout=3D1= 5000&retry=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.523; org.apache.solr.core.= CoreContainer; Registering Log Listener [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.523; org.apache.solr.core.= CoreContainer; Zookeeper client=3D127.0.0.1:40169/solr [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.524; org.apache.solr.cloud= .ZkController; zkHost includes chroot [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.525; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.526; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@ed8229 name:ZooKeeperConnection Watcher:127.0.0.1:40169 got event Wa= tchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.526; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.528; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D500&maxConnectionsPerHost=3D16&socketTimeout=3D120000&connTimeout=3D150= 00 [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.531; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.532; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@b1930f name:ZooKeeperConnection Watcher:127.0.0.1:40169/solr got eve= nt WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.532; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:45.535; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cluster state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.537; org.apache.solr.cloud= .ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:497= 41__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.538; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:49741__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.540; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.541; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.542; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.542; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.542; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.543; org.apache.solr.commo= n.cloud.ZkStateReader$3; Updating live nodes... (5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.547; org.apache.solr.servl= et.SolrDispatchFilter; user.dir=3D/mnt/ssd/jenkins/workspace/Lucene-Solr-4.= x-Linux/solr/build/solr-core/test/J1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.547; org.apache.solr.servl= et.SolrDispatchFilter; SolrDispatchFilter.init() done [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.548; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.551; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.556; org.apache.solr.handl= er.admin.CollectionsHandler; Creating Collection : numShards=3D2&name=3Dnod= es_used_collection&replicationFactor=3D2&action=3DCREATE&wt=3Djavabin&versi= on=3D2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.557; org.apache.solr.cloud= .DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/colle= ction-queue-work state: SyncConnected type NodeChildrenChanged [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.558; org.apache.solr.cloud= .OverseerCollectionProcessor; Overseer Collection Processor: Get the messag= e id:/overseer/collection-queue-work/qn-0000000000 message:{ [junit4:junit4] 1> "operation":"createcollection", [junit4:junit4] 1> "numShards":"2", [junit4:junit4] 1> "maxShardsPerNode":null, [junit4:junit4] 1> "createNodeSet":null, [junit4:junit4] 1> "name":"nodes_used_collection", [junit4:junit4] 1> "replicationFactor":"2"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.558; org.apache.solr.cloud= .OverseerCollectionProcessor; Creating shard nodes_used_collection_shard1_r= eplica1 as part of slice shard1 of collection nodes_used_collection on 127.= 0.0.1:58902__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.559; org.apache.solr.cloud= .OverseerCollectionProcessor; Creating shard nodes_used_collection_shard1_r= eplica2 as part of slice shard1 of collection nodes_used_collection on 127.= 0.0.1:60642__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.560; org.apache.solr.cloud= .OverseerCollectionProcessor; Creating shard nodes_used_collection_shard2_r= eplica1 as part of slice shard2 of collection nodes_used_collection on 127.= 0.0.1:49741__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.560; org.apache.solr.cloud= .OverseerCollectionProcessor; Creating shard nodes_used_collection_shard2_r= eplica2 as part of slice shard2 of collection nodes_used_collection on 127.= 0.0.1:51825__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.561; org.apache.solr.core.= CoreContainer; Creating SolrCore 'nodes_used_collection_shard1_replica2' us= ing instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/buil= d/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest= -jetty3-1366186244083/nodes_used_collection_shard1_replica2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.561; org.apache.solr.cloud= .ZkController; Check for collection zkNode:nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.561; org.apache.solr.core.= CoreContainer; Creating SolrCore 'nodes_used_collection_shard1_replica1' us= ing instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/buil= d/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest= -controljetty-1366186241479/nodes_used_collection_shard1_replica1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.562; org.apache.solr.cloud= .ZkController; Check for collection zkNode:nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.561; org.apache.solr.core.= CoreContainer; Creating SolrCore 'nodes_used_collection_shard2_replica1' us= ing instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/buil= d/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest= -jetty4-1366186245364/nodes_used_collection_shard2_replica1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.563; org.apache.solr.core.= CoreContainer; Creating SolrCore 'nodes_used_collection_shard2_replica2' us= ing instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/buil= d/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest= -jetty2-1366186242905/nodes_used_collection_shard2_replica2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.563; org.apache.solr.cloud= .ZkController; Check for collection zkNode:nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.563; org.apache.solr.cloud= .ZkController; Creating collection in ZooKeeper:nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.564; org.apache.solr.cloud= .ZkController; Looking for collection configName [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.565; org.apache.solr.cloud= .ZkController; Creating collection in ZooKeeper:nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.563; org.apache.solr.cloud= .ZkController; Creating collection in ZooKeeper:nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.563; org.apache.solr.cloud= .ZkController; Check for collection zkNode:nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.565; org.apache.solr.cloud= .ZkController; Only one config set found in zk - using it:conf1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.566; org.apache.solr.cloud= .ZkController; Creating collection in ZooKeeper:nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.565; org.apache.solr.cloud= .ZkController; Looking for collection configName [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.565; org.apache.solr.cloud= .ZkController; Looking for collection configName [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.566; org.apache.solr.cloud= .ZkController; Looking for collection configName [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.566; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.568; org.apache.solr.cloud= .ZkController; Only one config set found in zk - using it:conf1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.568; org.apache.solr.cloud= .ZkController; Only one config set found in zk - using it:conf1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.569; org.apache.solr.cloud= .ZkController; Only one config set found in zk - using it:conf1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.569; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.569; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.568; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.571; org.apache.solr.cloud= .ZkController; Load collection config from:/collections/nodes_used_collecti= on [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.572; org.apache.solr.cloud= .ZkController; Load collection config from:/collections/nodes_used_collecti= on [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.692; org.apache.solr.cloud= .ZkController; Load collection config from:/collections/nodes_used_collecti= on [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.693; org.apache.solr.cloud= .ZkController; Load collection config from:/collections/nodes_used_collecti= on [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.692; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244083/nodes_used_= collection_shard1_replica2/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.694; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245364/nodes_used_= collection_shard2_replica1/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.695; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242905/nodes_used_= collection_shard2_replica2/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.695; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366186241479/nodes= _used_collection_shard1_replica1/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.740; org.apache.solr.core.= SolrConfig; Using Lucene MatchVersion: LUCENE_43 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.758; org.apache.solr.core.= SolrConfig; Using Lucene MatchVersion: LUCENE_43 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.759; org.apache.solr.core.= SolrConfig; Using Lucene MatchVersion: LUCENE_43 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.775; org.apache.solr.core.= SolrConfig; Using Lucene MatchVersion: LUCENE_43 [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.805; org.apache.solr.core.= SolrConfig; Loaded SolrConfig: solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.819; org.apache.solr.schem= a.IndexSchema; Reading Solr Schema from schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.824; org.apache.solr.schem= a.IndexSchema; Schema name=3Dtest [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.829; org.apache.solr.core.= SolrConfig; Loaded SolrConfig: solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.831; org.apache.solr.schem= a.IndexSchema; Reading Solr Schema from schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.832; org.apache.solr.core.= SolrConfig; Loaded SolrConfig: solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.834; org.apache.solr.schem= a.IndexSchema; Reading Solr Schema from schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.837; org.apache.solr.schem= a.IndexSchema; Schema name=3Dtest [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.839; org.apache.solr.schem= a.IndexSchema; Schema name=3Dtest [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.886; org.apache.solr.core.= SolrConfig; Loaded SolrConfig: solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.888; org.apache.solr.schem= a.IndexSchema; Reading Solr Schema from schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:46.894; org.apache.solr.schem= a.IndexSchema; Schema name=3Dtest [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.496; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Initialized with rates=3Dopen-exchange-rate= s.json, refreshInterval=3D1440. [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.504; org.apache.solr.schem= a.IndexSchema; default search field in schema is text [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.507; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Initialized with rates=3Dopen-exchange-rate= s.json, refreshInterval=3D1440. [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.508; org.apache.solr.schem= a.IndexSchema; unique key field: id [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.512; org.apache.solr.schem= a.IndexSchema; default search field in schema is text [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.514; org.apache.solr.schem= a.IndexSchema; unique key field: id [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.516; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Initialized with rates=3Dopen-exchange-rate= s.json, refreshInterval=3D1440. [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.526; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.527; org.apache.solr.schem= a.IndexSchema; default search field in schema is text [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.529; org.apache.solr.schem= a.IndexSchema; unique key field: id [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.530; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.531; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.534; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.536; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.536; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.536; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.537; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.538; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.538; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.538; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.539; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Initialized with rates=3Dopen-exchange-rate= s.json, refreshInterval=3D1440. [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.540; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.541; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.539; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard2_replica2] Opening new SolrCore at /= mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J= 1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242= 905/nodes_used_collection_shard2_replica2/, dataDir=3D/mnt/ssd/jenkins/work= space/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.= cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242905/nodes_used_colle= ction_shard2_replica2/data/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.542; org.apache.solr.core.= JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Serve= r: com.sun.jmx.mbeanserver.JmxMBeanServer@196b20e [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.543; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.544; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.544; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.544; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty2-1366186242905/nodes_used_collectio= n_shard2_replica2/data [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.545; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.545; org.apache.solr.core.= SolrCore; New index directory detected: old=3Dnull new=3D/mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242905/nodes_used_co= llection_shard2_replica2/data/index/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.546; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.546; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard2_replica2] Solr index directory '/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-136618624290= 5/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating= new index... [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.547; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty2-1366186242905/nodes_used_collectio= n_shard2_replica2/data/index [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.547; org.apache.solr.schem= a.IndexSchema; default search field in schema is text [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.548; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.548; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.549; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.549; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.549; org.apache.solr.core.= SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=3D1 [junit4:junit4] 1> =09commit{dir=3DMockDirWrapper(org.apache.lucene.store= .RAMDirectory@1f17223 lockFactory=3Dorg.apache.lucene.store.NativeFSLockFac= tory@65a220),segFN=3Dsegments_1,generation=3D1,filenames=3D[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.550; org.apache.solr.core.= SolrDeletionPolicy; newest commit =3D 1[segments_1] [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.550; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.550; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.549; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard2_replica1] Opening new SolrCore at /= mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J= 1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245= 364/nodes_used_collection_shard2_replica1/, dataDir=3D/mnt/ssd/jenkins/work= space/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.= cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245364/nodes_used_colle= ction_shard2_replica1/data/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.551; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.551; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.551; org.apache.solr.schem= a.IndexSchema; unique key field: id [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.551; org.apache.solr.core.= JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Serve= r: com.sun.jmx.mbeanserver.JmxMBeanServer@196b20e [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.552; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.553; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.552; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.553; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.553; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.554; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard1_replica1] Opening new SolrCore at /= mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J= 1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366= 186241479/nodes_used_collection_shard1_replica1/, dataDir=3D/mnt/ssd/jenkin= s/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache= .solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366186241479/node= s_used_collection_shard1_replica1/data/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.555; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty4-1366186245364/nodes_used_collectio= n_shard2_replica1/data [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.554; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit= " [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.555; org.apache.solr.core.= SolrCore; New index directory detected: old=3Dnull new=3D/mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245364/nodes_used_co= llection_shard2_replica1/data/index/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.556; org.apache.solr.core.= RequestHandlers; created standard: solr.StandardRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.555; org.apache.solr.core.= JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Serve= r: com.sun.jmx.mbeanserver.JmxMBeanServer@196b20e [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.556; org.apache.solr.core.= RequestHandlers; created /get: solr.RealTimeGetHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.557; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.556; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard2_replica1] Solr index directory '/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-136618624536= 4/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating= new index... [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.557; org.apache.solr.core.= RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.557; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-controljetty-1366186241479/nodes_used_col= lection_shard1_replica1/data [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.558; org.apache.solr.core.= RequestHandlers; created /replication: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.558; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty4-1366186245364/nodes_used_collectio= n_shard2_replica1/data/index [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.558; org.apache.solr.core.= RequestHandlers; created /update: solr.UpdateRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.558; org.apache.solr.core.= SolrCore; New index directory detected: old=3Dnull new=3D/mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366186241479/nodes_u= sed_collection_shard1_replica1/data/index/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.559; org.apache.solr.core.= RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandle= rs [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.559; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard1_replica1] Solr index directory '/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-136618= 6241479/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Cr= eating new index... [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.560; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-controljetty-1366186241479/nodes_used_col= lection_shard1_replica1/data/index [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.560; org.apache.solr.core.= SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=3D1 [junit4:junit4] 1> =09commit{dir=3DMockDirWrapper(org.apache.lucene.store= .RAMDirectory@1ef78d2 lockFactory=3Dorg.apache.lucene.store.NativeFSLockFac= tory@1d9ac46),segFN=3Dsegments_1,generation=3D1,filenames=3D[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.561; org.apache.solr.core.= SolrDeletionPolicy; newest commit =3D 1[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.562; org.apache.solr.handl= er.loader.XMLLoader; xsltCacheLifetimeSeconds=3D60 [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.562; org.apache.solr.core.= SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=3D1 [junit4:junit4] 1> =09commit{dir=3DMockDirWrapper(org.apache.lucene.store= .RAMDirectory@154de8c lockFactory=3Dorg.apache.lucene.store.NativeFSLockFac= tory@1a683d5),segFN=3Dsegments_1,generation=3D1,filenames=3D[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.563; org.apache.solr.core.= SolrDeletionPolicy; newest commit =3D 1[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.563; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.564; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.565; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit= " [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.565; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.565; org.apache.solr.core.= RequestHandlers; created standard: solr.StandardRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.566; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.566; org.apache.solr.core.= RequestHandlers; created /get: solr.RealTimeGetHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.567; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit= " [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.567; org.apache.solr.core.= RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.567; org.apache.solr.core.= RequestHandlers; created standard: solr.StandardRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.567; org.apache.solr.core.= RequestHandlers; created /replication: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.568; org.apache.solr.core.= RequestHandlers; created /get: solr.RealTimeGetHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.568; org.apache.solr.core.= RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.568; org.apache.solr.core.= RequestHandlers; created /update: solr.UpdateRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.569; org.apache.solr.core.= RequestHandlers; created /replication: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.569; org.apache.solr.core.= RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandle= rs [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.570; org.apache.solr.core.= RequestHandlers; created /update: solr.UpdateRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.570; org.apache.solr.core.= RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandle= rs [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.571; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.573; org.apache.solr.handl= er.loader.XMLLoader; xsltCacheLifetimeSeconds=3D60 [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.573; org.apache.solr.handl= er.loader.XMLLoader; xsltCacheLifetimeSeconds=3D60 [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.576; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.579; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.580; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.580; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.580; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.581; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.581; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.582; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.582; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard1_replica2] Opening new SolrCore at /= mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J= 1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244= 083/nodes_used_collection_shard1_replica2/, dataDir=3D/mnt/ssd/jenkins/work= space/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.= cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244083/nodes_used_colle= ction_shard1_replica2/data/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.582; org.apache.solr.core.= JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Serve= r: com.sun.jmx.mbeanserver.JmxMBeanServer@196b20e [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.583; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.583; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty3-1366186244083/nodes_used_collectio= n_shard1_replica2/data [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.584; org.apache.solr.core.= SolrCore; New index directory detected: old=3Dnull new=3D/mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244083/nodes_used_co= llection_shard1_replica2/data/index/ [junit4:junit4] 1> WARN - 2013-04-17 08:10:47.584; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard1_replica2] Solr index directory '/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-136618624408= 3/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating= new index... [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.584; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty3-1366186244083/nodes_used_collectio= n_shard1_replica2/data/index [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.586; org.apache.solr.core.= SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=3D1 [junit4:junit4] 1> =09commit{dir=3DMockDirWrapper(org.apache.lucene.store= .RAMDirectory@1f6b9c6 lockFactory=3Dorg.apache.lucene.store.NativeFSLockFac= tory@2e062c),segFN=3Dsegments_1,generation=3D1,filenames=3D[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.586; org.apache.solr.core.= SolrDeletionPolicy; newest commit =3D 1[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.588; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.588; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.589; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit= " [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.589; org.apache.solr.core.= RequestHandlers; created standard: solr.StandardRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.590; org.apache.solr.core.= RequestHandlers; created /get: solr.RealTimeGetHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.590; org.apache.solr.core.= RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.590; org.apache.solr.core.= RequestHandlers; created /replication: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.591; org.apache.solr.core.= RequestHandlers; created /update: solr.UpdateRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.591; org.apache.solr.core.= RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandle= rs [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.593; org.apache.solr.handl= er.loader.XMLLoader; xsltCacheLifetimeSeconds=3D60 [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.615; org.apache.solr.searc= h.SolrIndexSearcher; Opening Searcher@4c0a0a main [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.615; org.apache.solr.updat= e.UpdateHandler; Clearing tlog files, tlogDir=3D/mnt/ssd/jenkins/workspace/= Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.= CollectionsAPIDistributedZkTest-jetty2-1366186242905/nodes_used_collection_= shard2_replica2/data/tlog [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.616; org.apache.solr.updat= e.CommitTracker; Hard AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.616; org.apache.solr.updat= e.CommitTracker; Soft AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.620; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard2_replica2] Registered new searcher S= earcher@4c0a0a main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.628; org.apache.solr.searc= h.SolrIndexSearcher; Opening Searcher@1dec2e6 main [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.628; org.apache.solr.cloud= .ZkController; publishing core=3Dnodes_used_collection_shard2_replica2 stat= e=3Ddown [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.628; org.apache.solr.updat= e.UpdateHandler; Clearing tlog files, tlogDir=3D/mnt/ssd/jenkins/workspace/= Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.= CollectionsAPIDistributedZkTest-controljetty-1366186241479/nodes_used_colle= ction_shard1_replica1/data/tlog [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.628; org.apache.solr.searc= h.SolrIndexSearcher; Opening Searcher@1cd8f19 main [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.629; org.apache.solr.updat= e.CommitTracker; Hard AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.629; org.apache.solr.updat= e.UpdateHandler; Clearing tlog files, tlogDir=3D/mnt/ssd/jenkins/workspace/= Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.= CollectionsAPIDistributedZkTest-jetty4-1366186245364/nodes_used_collection_= shard2_replica1/data/tlog [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.629; org.apache.solr.updat= e.CommitTracker; Soft AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.630; org.apache.solr.updat= e.CommitTracker; Hard AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.630; org.apache.solr.updat= e.CommitTracker; Soft AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.632; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard1_replica1] Registered new searcher S= earcher@1dec2e6 main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.634; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard2_replica1] Registered new searcher S= earcher@1cd8f19 main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.634; org.apache.solr.cloud= .ZkController; publishing core=3Dnodes_used_collection_shard1_replica1 stat= e=3Ddown [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.635; org.apache.solr.cloud= .ZkController; publishing core=3Dnodes_used_collection_shard2_replica1 stat= e=3Ddown [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.646; org.apache.solr.searc= h.SolrIndexSearcher; Opening Searcher@1c2cada main [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.647; org.apache.solr.updat= e.UpdateHandler; Clearing tlog files, tlogDir=3D/mnt/ssd/jenkins/workspace/= Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.= CollectionsAPIDistributedZkTest-jetty3-1366186244083/nodes_used_collection_= shard1_replica2/data/tlog [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.648; org.apache.solr.updat= e.CommitTracker; Hard AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.648; org.apache.solr.updat= e.CommitTracker; Soft AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.651; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard1_replica2] Registered new searcher S= earcher@1c2cada main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.652; org.apache.solr.cloud= .ZkController; publishing core=3Dnodes_used_collection_shard1_replica2 stat= e=3Ddown [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.724; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.725; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D2 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":null, [junit4:junit4] 1> "numShards":"2", [junit4:junit4] 1> "shard":"shard2", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"down", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"nodes_used_collection_shard2_replica2", [junit4:junit4] 1> "collection":"nodes_used_collection", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:51825__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:51825/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.725; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Create collection nodes_used_collection with= numShards 2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.728; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D2 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":null, [junit4:junit4] 1> "numShards":"2", [junit4:junit4] 1> "shard":"shard1", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"down", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"nodes_used_collection_shard1_replica1", [junit4:junit4] 1> "collection":"nodes_used_collection", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:58902__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:58902/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.730; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D2 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":null, [junit4:junit4] 1> "numShards":"2", [junit4:junit4] 1> "shard":"shard2", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"down", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"nodes_used_collection_shard2_replica1", [junit4:junit4] 1> "collection":"nodes_used_collection", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:49741__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:49741/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.732; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D2 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":null, [junit4:junit4] 1> "numShards":"2", [junit4:junit4] 1> "shard":"shard1", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"down", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"nodes_used_collection_shard1_replica2", [junit4:junit4] 1> "collection":"nodes_used_collection", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:60642__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:60642/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.736; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.736; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.736; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.736; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.736; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:47.736; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.629; org.apache.solr.core.= CoreContainer; registering core: nodes_used_collection_shard2_replica2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.630; org.apache.solr.cloud= .ZkController; Register replica - core:nodes_used_collection_shard2_replica= 2 address:http://127.0.0.1:51825/_r/eq collection:nodes_used_collection sha= rd:shard2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.630; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/nodes_used_collection/leader_e= lect/shard2/election [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.635; org.apache.solr.core.= CoreContainer; registering core: nodes_used_collection_shard1_replica1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.636; org.apache.solr.cloud= .ShardLeaderElectionContext; Running the leader process. [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.636; org.apache.solr.cloud= .ZkController; Register replica - core:nodes_used_collection_shard1_replica= 1 address:http://127.0.0.1:58902/_r/eq collection:nodes_used_collection sha= rd:shard1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.636; org.apache.solr.core.= CoreContainer; registering core: nodes_used_collection_shard2_replica1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.637; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/nodes_used_collection/leader_e= lect/shard1/election [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.637; org.apache.solr.cloud= .ZkController; Register replica - core:nodes_used_collection_shard2_replica= 1 address:http://127.0.0.1:49741/_r/eq collection:nodes_used_collection sha= rd:shard2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.638; org.apache.solr.cloud= .ShardLeaderElectionContext; Waiting until we see more replicas up: total= =3D2 found=3D1 timeoutin=3D179999 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.644; org.apache.solr.cloud= .ShardLeaderElectionContext; Running the leader process. [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.654; org.apache.solr.core.= CoreContainer; registering core: nodes_used_collection_shard1_replica2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.654; org.apache.solr.cloud= .ZkController; Register replica - core:nodes_used_collection_shard1_replica= 2 address:http://127.0.0.1:60642/_r/eq collection:nodes_used_collection sha= rd:shard1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.693; org.apache.solr.cloud= .ShardLeaderElectionContext; Enough replicas found to continue. [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.694; org.apache.solr.cloud= .ShardLeaderElectionContext; I may be the new leader - try and sync [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.694; org.apache.solr.cloud= .SyncStrategy; Sync replicas to http://127.0.0.1:58902/_r/eq/nodes_used_col= lection_shard1_replica1/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.694; org.apache.solr.updat= e.PeerSync; PeerSync: core=3Dnodes_used_collection_shard1_replica1 url=3Dht= tp://127.0.0.1:58902/_r/eq START replicas=3D[http://127.0.0.1:60642/_r/eq/n= odes_used_collection_shard1_replica2/] nUpdates=3D100 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.695; org.apache.solr.updat= e.PeerSync; PeerSync: core=3Dnodes_used_collection_shard1_replica1 url=3Dht= tp://127.0.0.1:58902/_r/eq DONE. We have no versions. sync failed. [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.695; org.apache.solr.cloud= .SyncStrategy; Leader's attempt to sync with shard failed, moving to the ne= xt canidate [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.695; org.apache.solr.cloud= .ShardLeaderElectionContext; We failed sync, but we have no versions - we c= an't sync in that case - we were active before, so become leader anyway [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.696; org.apache.solr.cloud= .ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:58902/_r= /eq/nodes_used_collection_shard1_replica1/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.696; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/nodes_used_collection/leaders/= shard1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:48.699; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard1_replica2] webapp=3D/_r/eq path=3D/g= et params=3D{getVersions=3D100&distrib=3Dfalse&wt=3Djavabin&qt=3D/get&versi= on=3D2} status=3D0 QTime=3D0=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.139; org.apache.solr.cloud= .ShardLeaderElectionContext; Enough replicas found to continue. [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.139; org.apache.solr.cloud= .ShardLeaderElectionContext; I may be the new leader - try and sync [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.140; org.apache.solr.cloud= .SyncStrategy; Sync replicas to http://127.0.0.1:51825/_r/eq/nodes_used_col= lection_shard2_replica2/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.140; org.apache.solr.updat= e.PeerSync; PeerSync: core=3Dnodes_used_collection_shard2_replica2 url=3Dht= tp://127.0.0.1:51825/_r/eq START replicas=3D[http://127.0.0.1:49741/_r/eq/n= odes_used_collection_shard2_replica1/] nUpdates=3D100 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.141; org.apache.solr.updat= e.PeerSync; PeerSync: core=3Dnodes_used_collection_shard2_replica2 url=3Dht= tp://127.0.0.1:51825/_r/eq DONE. We have no versions. sync failed. [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.141; org.apache.solr.cloud= .SyncStrategy; Leader's attempt to sync with shard failed, moving to the ne= xt canidate [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.142; org.apache.solr.cloud= .ShardLeaderElectionContext; We failed sync, but we have no versions - we c= an't sync in that case - we were active before, so become leader anyway [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.142; org.apache.solr.cloud= .ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:51825/_r= /eq/nodes_used_collection_shard2_replica2/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.142; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/nodes_used_collection/leaders/= shard2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.145; org.apache.solr.core.= SolrCore; [nodes_used_collection_shard2_replica1] webapp=3D/_r/eq path=3D/g= et params=3D{getVersions=3D100&distrib=3Dfalse&wt=3Djavabin&qt=3D/get&versi= on=3D2} status=3D0 QTime=3D1=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.244; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.255; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.255; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.255; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.255; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.255; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.255; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.298; org.apache.solr.cloud= .ZkController; We are http://127.0.0.1:51825/_r/eq/nodes_used_collection_sh= ard2_replica2/ and leader is http://127.0.0.1:51825/_r/eq/nodes_used_collec= tion_shard2_replica2/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.298; org.apache.solr.cloud= .ZkController; No LogReplay needed for core=3Dnodes_used_collection_shard2_= replica2 baseURL=3Dhttp://127.0.0.1:51825/_r/eq [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.299; org.apache.solr.cloud= .ZkController; I am the leader, no recovery necessary [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.299; org.apache.solr.cloud= .ZkController; publishing core=3Dnodes_used_collection_shard2_replica2 stat= e=3Dactive [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.301; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.302; org.apache.solr.core.= CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene= -Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Collec= tionsAPIDistributedZkTest-jetty2-1366186242905/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.302; org.apache.solr.cloud= .ZkController; We are http://127.0.0.1:58902/_r/eq/nodes_used_collection_sh= ard1_replica1/ and leader is http://127.0.0.1:58902/_r/eq/nodes_used_collec= tion_shard1_replica1/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.303; org.apache.solr.core.= SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lu= cene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Co= llectionsAPIDistributedZkTest-jetty2-1366186242905/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.304; org.apache.solr.cloud= .ZkController; No LogReplay needed for core=3Dnodes_used_collection_shard1_= replica1 baseURL=3Dhttp://127.0.0.1:58902/_r/eq [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.304; org.apache.solr.cloud= .ZkController; I am the leader, no recovery necessary [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.304; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= numShards=3D2&shard=3Dshard2&name=3Dnodes_used_collection_shard2_replica2&a= ction=3DCREATE&collection=3Dnodes_used_collection&wt=3Djavabin&qt=3D/admin/= cores&version=3D2} status=3D0 QTime=3D2741=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.304; org.apache.solr.cloud= .ZkController; publishing core=3Dnodes_used_collection_shard1_replica1 stat= e=3Dactive [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.306; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.306; org.apache.solr.core.= CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene= -Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Collec= tionsAPIDistributedZkTest-controljetty-1366186241479/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.307; org.apache.solr.core.= SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lu= cene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Co= llectionsAPIDistributedZkTest-controljetty-1366186241479/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.308; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= numShards=3D2&shard=3Dshard1&name=3Dnodes_used_collection_shard1_replica1&a= ction=3DCREATE&collection=3Dnodes_used_collection&wt=3Djavabin&qt=3D/admin/= cores&version=3D2} status=3D0 QTime=3D2747=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.640; org.apache.solr.cloud= .ZkController; We are http://127.0.0.1:49741/_r/eq/nodes_used_collection_sh= ard2_replica1/ and leader is http://127.0.0.1:51825/_r/eq/nodes_used_collec= tion_shard2_replica2/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.641; org.apache.solr.cloud= .ZkController; No LogReplay needed for core=3Dnodes_used_collection_shard2_= replica1 baseURL=3Dhttp://127.0.0.1:49741/_r/eq [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.641; org.apache.solr.cloud= .ZkController; Core needs to recover:nodes_used_collection_shard2_replica1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.641; org.apache.solr.updat= e.DefaultSolrCoreState; Running recovery - first canceling any ongoing reco= very [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.642; org.apache.solr.cloud= .RecoveryStrategy; Starting recovery process. core=3Dnodes_used_collection= _shard2_replica1 recoveringAfterStartup=3Dtrue [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.642; org.apache.solr.cloud= .RecoveryStrategy; ###### startupVersions=3D[] [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.642; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.642; org.apache.solr.cloud= .ZkController; publishing core=3Dnodes_used_collection_shard2_replica1 stat= e=3Drecovering [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.643; org.apache.solr.core.= CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene= -Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Collec= tionsAPIDistributedZkTest-jetty4-1366186245364/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.643; org.apache.solr.core.= SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lu= cene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Co= llectionsAPIDistributedZkTest-jetty4-1366186245364/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.644; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.644; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= numShards=3D2&shard=3Dshard2&name=3Dnodes_used_collection_shard2_replica1&a= ction=3DCREATE&collection=3Dnodes_used_collection&wt=3Djavabin&qt=3D/admin/= cores&version=3D2} status=3D0 QTime=3D3083=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.647; org.apache.solr.handl= er.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:49741_= _r%2Feq_nodes_used_collection_shard2_replica1, state: recovering, checkLive= : true, onlyIfLeader: true [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.695; org.apache.solr.cloud= .ZkController; We are http://127.0.0.1:60642/_r/eq/nodes_used_collection_sh= ard1_replica2/ and leader is http://127.0.0.1:58902/_r/eq/nodes_used_collec= tion_shard1_replica1/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.696; org.apache.solr.cloud= .ZkController; No LogReplay needed for core=3Dnodes_used_collection_shard1_= replica2 baseURL=3Dhttp://127.0.0.1:60642/_r/eq [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.696; org.apache.solr.cloud= .ZkController; Core needs to recover:nodes_used_collection_shard1_replica2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.696; org.apache.solr.updat= e.DefaultSolrCoreState; Running recovery - first canceling any ongoing reco= very [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.697; org.apache.solr.cloud= .RecoveryStrategy; Starting recovery process. core=3Dnodes_used_collection= _shard1_replica2 recoveringAfterStartup=3Dtrue [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.697; org.apache.solr.cloud= .RecoveryStrategy; ###### startupVersions=3D[] [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.697; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.698; org.apache.solr.cloud= .ZkController; publishing core=3Dnodes_used_collection_shard1_replica2 stat= e=3Drecovering [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.698; org.apache.solr.core.= CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene= -Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Collec= tionsAPIDistributedZkTest-jetty3-1366186244083/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.699; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.699; org.apache.solr.core.= SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lu= cene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Co= llectionsAPIDistributedZkTest-jetty3-1366186244083/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.700; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= numShards=3D2&shard=3Dshard1&name=3Dnodes_used_collection_shard1_replica2&a= ction=3DCREATE&collection=3Dnodes_used_collection&wt=3Djavabin&qt=3D/admin/= cores&version=3D2} status=3D0 QTime=3D3139=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.701; org.apache.solr.cloud= .OverseerCollectionProcessor; Finished create command on all shards for col= lection: nodes_used_collection [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.702; org.apache.solr.handl= er.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:60642_= _r%2Feq_nodes_used_collection_shard1_replica2, state: recovering, checkLive= : true, onlyIfLeader: true [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.762; org.apache.solr.cloud= .DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/colle= ction-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.764; org.apache.solr.cloud= .DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/colle= ction-queue-work state: SyncConnected type NodeChildrenChanged [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.764; org.apache.solr.cloud= .OverseerCollectionProcessor; Overseer Collection Processor: Message id:/ov= erseer/collection-queue-work/qn-0000000000 complete, response:{success=3D{n= ull=3D{responseHeader=3D{status=3D0,QTime=3D2741},core=3Dnodes_used_collect= ion_shard2_replica2,saved=3D/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linu= x/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistri= butedZkTest-jetty2-1366186242905/solr.xml},null=3D{responseHeader=3D{status= =3D0,QTime=3D2747},core=3Dnodes_used_collection_shard1_replica1,saved=3D/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-136618= 6241479/solr.xml},null=3D{responseHeader=3D{status=3D0,QTime=3D3083},core= =3Dnodes_used_collection_shard2_replica1,saved=3D/mnt/ssd/jenkins/workspace= /Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud= .CollectionsAPIDistributedZkTest-jetty4-1366186245364/solr.xml},null=3D{res= ponseHeader=3D{status=3D0,QTime=3D3139},core=3Dnodes_used_collection_shard1= _replica2,saved=3D/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/bui= ld/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTes= t-jetty3-1366186244083/solr.xml}}} [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.765; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/collections para= ms=3D{numShards=3D2&name=3Dnodes_used_collection&replicationFactor=3D2&acti= on=3DCREATE&wt=3Djavabin&version=3D2} status=3D0 QTime=3D3209=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.766; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config: [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.767; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.768; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@176da46 name:ZooKeeperConnection Watcher:127.0.0.1:40169/solr got ev= ent WatchedEvent state:SyncConnected type:None path:null path:null type:Non= e [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.768; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.769; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cluster state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.771; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config: [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.772; org.apache.solr.commo= n.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.773; org.apache.solr.commo= n.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionM= anager@1f97309 name:ZooKeeperConnection Watcher:127.0.0.1:40169/solr got ev= ent WatchedEvent state:SyncConnected type:None path:null path:null type:Non= e [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.774; org.apache.solr.commo= n.cloud.ConnectionManager; Client is connected to ZooKeeper [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.774; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cluster state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.779; org.apache.solr.handl= er.admin.CollectionsHandler; Creating Collection : numShards=3D1&maxShardsP= erNode=3D2&name=3Dawholynewcollection_0&replicationFactor=3D5&action=3DCREA= TE&wt=3Djavabin&version=3D2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.780; org.apache.solr.cloud= .DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/colle= ction-queue-work state: SyncConnected type NodeChildrenChanged [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.781; org.apache.solr.cloud= .OverseerCollectionProcessor; Overseer Collection Processor: Get the messag= e id:/overseer/collection-queue-work/qn-0000000002 message:{ [junit4:junit4] 1> "operation":"createcollection", [junit4:junit4] 1> "numShards":"1", [junit4:junit4] 1> "maxShardsPerNode":"2", [junit4:junit4] 1> "createNodeSet":null, [junit4:junit4] 1> "name":"awholynewcollection_0", [junit4:junit4] 1> "replicationFactor":"5"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.781; org.apache.solr.cloud= .OverseerCollectionProcessor; Creating shard awholynewcollection_0_shard1_r= eplica1 as part of slice shard1 of collection awholynewcollection_0 on 127.= 0.0.1:58902__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.781; org.apache.solr.cloud= .OverseerCollectionProcessor; Creating shard awholynewcollection_0_shard1_r= eplica2 as part of slice shard1 of collection awholynewcollection_0 on 127.= 0.0.1:51825__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.781; org.apache.solr.cloud= .OverseerCollectionProcessor; Creating shard awholynewcollection_0_shard1_r= eplica3 as part of slice shard1 of collection awholynewcollection_0 on 127.= 0.0.1:49741__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.781; org.apache.solr.cloud= .OverseerCollectionProcessor; Creating shard awholynewcollection_0_shard1_r= eplica4 as part of slice shard1 of collection awholynewcollection_0 on 127.= 0.0.1:60642__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.782; org.apache.solr.cloud= .OverseerCollectionProcessor; Creating shard awholynewcollection_0_shard1_r= eplica5 as part of slice shard1 of collection awholynewcollection_0 on 127.= 0.0.1:60700__r%2Feq [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.783; org.apache.solr.core.= CoreContainer; Creating SolrCore 'awholynewcollection_0_shard1_replica5' us= ing instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/buil= d/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest= -jetty1-1366186241726/awholynewcollection_0_shard1_replica5 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.783; org.apache.solr.core.= CoreContainer; Creating SolrCore 'awholynewcollection_0_shard1_replica3' us= ing instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/buil= d/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest= -jetty4-1366186245364/awholynewcollection_0_shard1_replica3 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.783; org.apache.solr.cloud= .ZkController; Check for collection zkNode:awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.783; org.apache.solr.core.= CoreContainer; Creating SolrCore 'awholynewcollection_0_shard1_replica1' us= ing instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/buil= d/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest= -controljetty-1366186241479/awholynewcollection_0_shard1_replica1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.784; org.apache.solr.cloud= .ZkController; Creating collection in ZooKeeper:awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.783; org.apache.solr.core.= CoreContainer; Creating SolrCore 'awholynewcollection_0_shard1_replica2' us= ing instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/buil= d/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest= -jetty2-1366186242905/awholynewcollection_0_shard1_replica2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.785; org.apache.solr.cloud= .ZkController; Check for collection zkNode:awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.784; org.apache.solr.cloud= .ZkController; Looking for collection configName [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.784; org.apache.solr.cloud= .ZkController; Check for collection zkNode:awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.783; org.apache.solr.core.= CoreContainer; Creating SolrCore 'awholynewcollection_0_shard1_replica4' us= ing instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/buil= d/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest= -jetty3-1366186244083/awholynewcollection_0_shard1_replica4 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.786; org.apache.solr.cloud= .ZkController; Creating collection in ZooKeeper:awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.783; org.apache.solr.cloud= .ZkController; Check for collection zkNode:awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.786; org.apache.solr.cloud= .ZkController; Looking for collection configName [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.786; org.apache.solr.cloud= .ZkController; Check for collection zkNode:awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.786; org.apache.solr.cloud= .ZkController; Only one config set found in zk - using it:conf1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.785; org.apache.solr.cloud= .ZkController; Creating collection in ZooKeeper:awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.788; org.apache.solr.cloud= .ZkController; Only one config set found in zk - using it:conf1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.788; org.apache.solr.cloud= .ZkController; Creating collection in ZooKeeper:awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.787; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.787; org.apache.solr.cloud= .ZkController; Creating collection in ZooKeeper:awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.788; org.apache.solr.cloud= .ZkController; Looking for collection configName [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.788; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.788; org.apache.solr.cloud= .ZkController; Looking for collection configName [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.789; org.apache.solr.cloud= .ZkController; Looking for collection configName [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.791; org.apache.solr.cloud= .ZkController; Only one config set found in zk - using it:conf1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.791; org.apache.solr.cloud= .ZkController; Load collection config from:/collections/awholynewcollection= _0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.792; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.791; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.792; org.apache.solr.cloud= .ZkController; Load collection config from:/collections/awholynewcollection= _0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.792; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.793; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245364/awholynewco= llection_0_shard1_replica3/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.793; org.apache.solr.cloud= .ZkController; Load collection config from:/collections/awholynewcollection= _0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.794; org.apache.solr.cloud= .ZkController; Load collection config from:/collections/awholynewcollection= _0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.794; org.apache.solr.cloud= .ZkController; Load collection config from:/collections/awholynewcollection= _0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.795; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242905/awholynewco= llection_0_shard1_replica2/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.796; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1366186241726/awholynewco= llection_0_shard1_replica5/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.795; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366186241479/awhol= ynewcollection_0_shard1_replica1/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.795; org.apache.solr.core.= SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins= /workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.= solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244083/awholynewco= llection_0_shard1_replica4/' [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.849; org.apache.solr.core.= SolrConfig; Using Lucene MatchVersion: LUCENE_43 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.854; org.apache.solr.core.= SolrConfig; Using Lucene MatchVersion: LUCENE_43 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.854; org.apache.solr.core.= SolrConfig; Using Lucene MatchVersion: LUCENE_43 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.859; org.apache.solr.core.= SolrConfig; Using Lucene MatchVersion: LUCENE_43 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.872; org.apache.solr.core.= SolrConfig; Using Lucene MatchVersion: LUCENE_43 [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.895; org.apache.solr.core.= SolrConfig; Loaded SolrConfig: solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.897; org.apache.solr.schem= a.IndexSchema; Reading Solr Schema from schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.900; org.apache.solr.schem= a.IndexSchema; Schema name=3Dtest [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.919; org.apache.solr.core.= SolrConfig; Loaded SolrConfig: solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.920; org.apache.solr.schem= a.IndexSchema; Reading Solr Schema from schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.932; org.apache.solr.schem= a.IndexSchema; Schema name=3Dtest [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.933; org.apache.solr.core.= SolrConfig; Loaded SolrConfig: solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.934; org.apache.solr.schem= a.IndexSchema; Reading Solr Schema from schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.935; org.apache.solr.core.= SolrConfig; Loaded SolrConfig: solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.939; org.apache.solr.schem= a.IndexSchema; Schema name=3Dtest [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.944; org.apache.solr.schem= a.IndexSchema; Reading Solr Schema from schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.954; org.apache.solr.core.= SolrConfig; Loaded SolrConfig: solrconfig.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.955; org.apache.solr.schem= a.IndexSchema; Schema name=3Dtest [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.956; org.apache.solr.schem= a.IndexSchema; Reading Solr Schema from schema.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:49.961; org.apache.solr.schem= a.IndexSchema; Schema name=3Dtest [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.563; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Initialized with rates=3Dopen-exchange-rate= s.json, refreshInterval=3D1440. [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.577; org.apache.solr.schem= a.IndexSchema; default search field in schema is text [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.581; org.apache.solr.schem= a.IndexSchema; unique key field: id [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.598; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.602; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.603; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Initialized with rates=3Dopen-exchange-rate= s.json, refreshInterval=3D1440. [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.612; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.613; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.614; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.614; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.614; org.apache.solr.schem= a.IndexSchema; default search field in schema is text [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.614; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Initialized with rates=3Dopen-exchange-rate= s.json, refreshInterval=3D1440. [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.615; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.616; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.616; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.616; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica5] Opening new SolrCore at /= mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J= 1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1366186241= 726/awholynewcollection_0_shard1_replica5/, dataDir=3D/mnt/ssd/jenkins/work= space/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.= cloud.CollectionsAPIDistributedZkTest-jetty1-1366186241726/awholynewcollect= ion_0_shard1_replica5/data/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.617; org.apache.solr.core.= JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Serve= r: com.sun.jmx.mbeanserver.JmxMBeanServer@196b20e [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.617; org.apache.solr.schem= a.IndexSchema; unique key field: id [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.617; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.618; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty1-1366186241726/awholynewcollection_= 0_shard1_replica5/data [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.619; org.apache.solr.core.= SolrCore; New index directory detected: old=3Dnull new=3D/mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty1-1366186241726/awholynewcoll= ection_0_shard1_replica5/data/index/ [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.619; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica5] Solr index directory '/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-136618624172= 6/awholynewcollection_0_shard1_replica5/data/index' doesn't exist. Creating= new index... [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.620; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty1-1366186241726/awholynewcollection_= 0_shard1_replica5/data/index [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.620; org.apache.solr.schem= a.IndexSchema; default search field in schema is text [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.622; org.apache.solr.core.= SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=3D1 [junit4:junit4] 1> =09commit{dir=3DMockDirWrapper(org.apache.lucene.store= .RAMDirectory@140a2db lockFactory=3Dorg.apache.lucene.store.NativeFSLockFac= tory@181f2cc),segFN=3Dsegments_1,generation=3D1,filenames=3D[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.623; org.apache.solr.core.= SolrDeletionPolicy; newest commit =3D 1[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.624; org.apache.solr.schem= a.IndexSchema; unique key field: id [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.625; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.626; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.626; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit= " [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.627; org.apache.solr.core.= RequestHandlers; created standard: solr.StandardRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.628; org.apache.solr.core.= RequestHandlers; created /get: solr.RealTimeGetHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.628; org.apache.solr.core.= RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.629; org.apache.solr.core.= RequestHandlers; created /replication: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.629; org.apache.solr.core.= RequestHandlers; created /update: solr.UpdateRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.629; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.630; org.apache.solr.core.= RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandle= rs [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.632; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Initialized with rates=3Dopen-exchange-rate= s.json, refreshInterval=3D1440. [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.634; org.apache.solr.handl= er.loader.XMLLoader; xsltCacheLifetimeSeconds=3D60 [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.634; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.635; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.644; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.644; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.646; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.646; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.646; org.apache.solr.schem= a.IndexSchema; default search field in schema is text [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.647; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.648; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.648; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.648; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.649; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.649; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica3] Opening new SolrCore at /= mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J= 1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245= 364/awholynewcollection_0_shard1_replica3/, dataDir=3D/mnt/ssd/jenkins/work= space/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.= cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245364/awholynewcollect= ion_0_shard1_replica3/data/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.649; org.apache.solr.schem= a.IndexSchema; unique key field: id [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.650; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.650; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.650; org.apache.solr.core.= JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Serve= r: com.sun.jmx.mbeanserver.JmxMBeanServer@196b20e [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.650; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.651; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.652; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty4-1366186245364/awholynewcollection_= 0_shard1_replica3/data [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.652; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.652; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.652; org.apache.solr.core.= SolrCore; New index directory detected: old=3Dnull new=3D/mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty4-1366186245364/awholynewcoll= ection_0_shard1_replica3/data/index/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.652; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.653; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica3] Solr index directory '/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-136618624536= 4/awholynewcollection_0_shard1_replica3/data/index' doesn't exist. Creating= new index... [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.653; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica1] Opening new SolrCore at /= mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J= 1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366= 186241479/awholynewcollection_0_shard1_replica1/, dataDir=3D/mnt/ssd/jenkin= s/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache= .solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366186241479/awho= lynewcollection_0_shard1_replica1/data/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.654; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty4-1366186245364/awholynewcollection_= 0_shard1_replica3/data/index [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.654; org.apache.solr.core.= JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Serve= r: com.sun.jmx.mbeanserver.JmxMBeanServer@196b20e [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.655; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.656; org.apache.solr.core.= SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=3D1 [junit4:junit4] 1> =09commit{dir=3DMockDirWrapper(org.apache.lucene.store= .RAMDirectory@b03bdf lockFactory=3Dorg.apache.lucene.store.NativeFSLockFact= ory@a2d8f6),segFN=3Dsegments_1,generation=3D1,filenames=3D[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.656; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Initialized with rates=3Dopen-exchange-rate= s.json, refreshInterval=3D1440. [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.656; org.apache.solr.core.= SolrDeletionPolicy; newest commit =3D 1[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.656; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-controljetty-1366186241479/awholynewcolle= ction_0_shard1_replica1/data [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.657; org.apache.solr.core.= SolrCore; New index directory detected: old=3Dnull new=3D/mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-controljetty-1366186241479/awholyn= ewcollection_0_shard1_replica1/data/index/ [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.657; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica1] Solr index directory '/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-136618= 6241479/awholynewcollection_0_shard1_replica1/data/index' doesn't exist. Cr= eating new index... [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.658; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-controljetty-1366186241479/awholynewcolle= ction_0_shard1_replica1/data/index [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.658; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.659; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.659; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.660; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit= " [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.660; org.apache.solr.core.= SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=3D1 [junit4:junit4] 1> =09commit{dir=3DMockDirWrapper(org.apache.lucene.store= .RAMDirectory@140f10a lockFactory=3Dorg.apache.lucene.store.NativeFSLockFac= tory@d3cf82),segFN=3Dsegments_1,generation=3D1,filenames=3D[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.660; org.apache.solr.core.= RequestHandlers; created standard: solr.StandardRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.660; org.apache.solr.core.= SolrDeletionPolicy; newest commit =3D 1[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.661; org.apache.solr.core.= RequestHandlers; created /get: solr.RealTimeGetHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.661; org.apache.solr.core.= RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.662; org.apache.solr.core.= RequestHandlers; created /replication: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.662; org.apache.solr.core.= RequestHandlers; created /update: solr.UpdateRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.663; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.663; org.apache.solr.core.= RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandle= rs [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.663; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.664; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.664; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit= " [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.665; org.apache.solr.schem= a.IndexSchema; default search field in schema is text [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.665; org.apache.solr.core.= RequestHandlers; created standard: solr.StandardRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.666; org.apache.solr.core.= RequestHandlers; created /get: solr.RealTimeGetHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.666; org.apache.solr.core.= RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.666; org.apache.solr.handl= er.loader.XMLLoader; xsltCacheLifetimeSeconds=3D60 [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.667; org.apache.solr.core.= RequestHandlers; created /replication: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.667; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.667; org.apache.solr.core.= RequestHandlers; created /update: solr.UpdateRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.668; org.apache.solr.core.= RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandle= rs [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.668; org.apache.solr.schem= a.IndexSchema; unique key field: id [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.668; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.669; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.669; org.apache.solr.handl= er.loader.XMLLoader; xsltCacheLifetimeSeconds=3D60 [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.669; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.670; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.671; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.671; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.671; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica2] Opening new SolrCore at /= mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J= 1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242= 905/awholynewcollection_0_shard1_replica2/, dataDir=3D/mnt/ssd/jenkins/work= space/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.= cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242905/awholynewcollect= ion_0_shard1_replica2/data/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.672; org.apache.solr.core.= JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Serve= r: com.sun.jmx.mbeanserver.JmxMBeanServer@196b20e [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.672; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.673; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty2-1366186242905/awholynewcollection_= 0_shard1_replica2/data [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.673; org.apache.solr.core.= SolrCore; New index directory detected: old=3Dnull new=3D/mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty2-1366186242905/awholynewcoll= ection_0_shard1_replica2/data/index/ [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.673; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica2] Solr index directory '/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-136618624290= 5/awholynewcollection_0_shard1_replica2/data/index' doesn't exist. Creating= new index... [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.674; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty2-1366186242905/awholynewcollection_= 0_shard1_replica2/data/index [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.675; org.apache.solr.core.= SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=3D1 [junit4:junit4] 1> =09commit{dir=3DMockDirWrapper(org.apache.lucene.store= .RAMDirectory@278040 lockFactory=3Dorg.apache.lucene.store.NativeFSLockFact= ory@95c753),segFN=3Dsegments_1,generation=3D1,filenames=3D[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.675; org.apache.solr.core.= SolrDeletionPolicy; newest commit =3D 1[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.677; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.677; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.678; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit= " [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.678; org.apache.solr.core.= RequestHandlers; created standard: solr.StandardRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.679; org.apache.solr.core.= RequestHandlers; created /get: solr.RealTimeGetHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.679; org.apache.solr.core.= RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.679; org.apache.solr.core.= RequestHandlers; created /replication: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.680; org.apache.solr.core.= RequestHandlers; created /update: solr.UpdateRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.680; org.apache.solr.core.= RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandle= rs [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.683; org.apache.solr.handl= er.loader.XMLLoader; xsltCacheLifetimeSeconds=3D60 [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.685; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.685; org.apache.solr.searc= h.SolrIndexSearcher; Opening Searcher@d84fcc main [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.686; org.apache.solr.updat= e.UpdateHandler; Clearing tlog files, tlogDir=3D/mnt/ssd/jenkins/workspace/= Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.= CollectionsAPIDistributedZkTest-jetty2-1366186242905/awholynewcollection_0_= shard1_replica2/data/tlog [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.686; org.apache.solr.searc= h.SolrIndexSearcher; Opening Searcher@1ffa535 main [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.686; org.apache.solr.updat= e.CommitTracker; Hard AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.687; org.apache.solr.updat= e.UpdateHandler; Clearing tlog files, tlogDir=3D/mnt/ssd/jenkins/workspace/= Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.= CollectionsAPIDistributedZkTest-jetty1-1366186241726/awholynewcollection_0_= shard1_replica5/data/tlog [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.687; org.apache.solr.updat= e.CommitTracker; Soft AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.688; org.apache.solr.updat= e.CommitTracker; Hard AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.688; org.apache.solr.updat= e.CommitTracker; Soft AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.691; org.apache.solr.schem= a.FileExchangeRateProvider; Reloading exchange rates from file currency.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.692; org.apache.solr.cloud= .ZkController; publishing core=3Dawholynewcollection_0_shard1_replica2 stat= e=3Ddown [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.693; org.apache.solr.cloud= .ZkController; publishing core=3Dawholynewcollection_0_shard1_replica5 stat= e=3Ddown [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.693; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica2] Registered new searcher S= earcher@d84fcc main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.694; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica5] Registered new searcher S= earcher@1ffa535 main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.697; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.698; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.698; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.698; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange= -rates.json [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.699; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOT= E [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.700; org.apache.solr.schem= a.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.700; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.700; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica4] Opening new SolrCore at /= mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J= 1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244= 083/awholynewcollection_0_shard1_replica4/, dataDir=3D/mnt/ssd/jenkins/work= space/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.= cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244083/awholynewcollect= ion_0_shard1_replica4/data/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.701; org.apache.solr.core.= JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Serve= r: com.sun.jmx.mbeanserver.JmxMBeanServer@196b20e [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.701; org.apache.solr.core.= SolrCore; org.apache.solr.core.MockDirectoryFactory [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.702; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty3-1366186244083/awholynewcollection_= 0_shard1_replica4/data [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.702; org.apache.solr.core.= SolrCore; New index directory detected: old=3Dnull new=3D/mnt/ssd/jenkins/w= orkspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.so= lr.cloud.CollectionsAPIDistributedZkTest-jetty3-1366186244083/awholynewcoll= ection_0_shard1_replica4/data/index/ [junit4:junit4] 1> WARN - 2013-04-17 08:10:50.702; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica4] Solr index directory '/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-136618624408= 3/awholynewcollection_0_shard1_replica4/data/index' doesn't exist. Creating= new index... [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.703; org.apache.solr.core.= CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspac= e/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.clou= d.CollectionsAPIDistributedZkTest-jetty3-1366186244083/awholynewcollection_= 0_shard1_replica4/data/index [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.704; org.apache.solr.core.= SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=3D1 [junit4:junit4] 1> =09commit{dir=3DMockDirWrapper(org.apache.lucene.store= .RAMDirectory@81c342 lockFactory=3Dorg.apache.lucene.store.NativeFSLockFact= ory@1e15800),segFN=3Dsegments_1,generation=3D1,filenames=3D[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.704; org.apache.solr.core.= SolrDeletionPolicy; newest commit =3D 1[segments_1] [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.706; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "dedupe" [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.706; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "stored_sig" [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.707; org.apache.solr.updat= e.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcess= orFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit= " [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.707; org.apache.solr.core.= RequestHandlers; created standard: solr.StandardRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.708; org.apache.solr.core.= RequestHandlers; created /get: solr.RealTimeGetHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.708; org.apache.solr.core.= RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.708; org.apache.solr.core.= RequestHandlers; created /replication: solr.ReplicationHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.709; org.apache.solr.core.= RequestHandlers; created /update: solr.UpdateRequestHandler [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.709; org.apache.solr.core.= RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandle= rs [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.711; org.apache.solr.handl= er.loader.XMLLoader; xsltCacheLifetimeSeconds=3D60 [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.719; org.apache.solr.searc= h.SolrIndexSearcher; Opening Searcher@1813dda main [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.719; org.apache.solr.updat= e.UpdateHandler; Clearing tlog files, tlogDir=3D/mnt/ssd/jenkins/workspace/= Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.= CollectionsAPIDistributedZkTest-jetty4-1366186245364/awholynewcollection_0_= shard1_replica3/data/tlog [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.720; org.apache.solr.updat= e.CommitTracker; Hard AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.720; org.apache.solr.updat= e.CommitTracker; Soft AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.721; org.apache.solr.searc= h.SolrIndexSearcher; Opening Searcher@faeb47 main [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.722; org.apache.solr.updat= e.UpdateHandler; Clearing tlog files, tlogDir=3D/mnt/ssd/jenkins/workspace/= Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.= CollectionsAPIDistributedZkTest-controljetty-1366186241479/awholynewcollect= ion_0_shard1_replica1/data/tlog [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.722; org.apache.solr.updat= e.CommitTracker; Hard AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.722; org.apache.solr.updat= e.CommitTracker; Soft AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.723; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica3] Registered new searcher S= earcher@1813dda main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.733; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica1] Registered new searcher S= earcher@faeb47 main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.733; org.apache.solr.cloud= .ZkController; publishing core=3Dawholynewcollection_0_shard1_replica3 stat= e=3Ddown [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.734; org.apache.solr.cloud= .ZkController; publishing core=3Dawholynewcollection_0_shard1_replica1 stat= e=3Ddown [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.764; org.apache.solr.searc= h.SolrIndexSearcher; Opening Searcher@194282 main [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.764; org.apache.solr.updat= e.UpdateHandler; Clearing tlog files, tlogDir=3D/mnt/ssd/jenkins/workspace/= Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.= CollectionsAPIDistributedZkTest-jetty3-1366186244083/awholynewcollection_0_= shard1_replica4/data/tlog [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.765; org.apache.solr.updat= e.CommitTracker; Hard AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.765; org.apache.solr.updat= e.CommitTracker; Soft AutoCommit: disabled [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.768; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica4] Registered new searcher S= earcher@194282 main{StandardDirectoryReader(segments_1:1)} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.770; org.apache.solr.cloud= .ZkController; publishing core=3Dawholynewcollection_0_shard1_replica4 stat= e=3Ddown [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.793; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.794; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D2 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":"127.0.0.1:51825__r%2Feq_nodes_used= _collection_shard2_replica2", [junit4:junit4] 1> "numShards":"2", [junit4:junit4] 1> "shard":"shard2", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"active", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"nodes_used_collection_shard2_replica2", [junit4:junit4] 1> "collection":"nodes_used_collection", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:51825__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:51825/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.796; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D2 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":"127.0.0.1:58902__r%2Feq_nodes_used= _collection_shard1_replica1", [junit4:junit4] 1> "numShards":"2", [junit4:junit4] 1> "shard":"shard1", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"active", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"nodes_used_collection_shard1_replica1", [junit4:junit4] 1> "collection":"nodes_used_collection", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:58902__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:58902/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.799; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D2 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":"127.0.0.1:49741__r%2Feq_nodes_used= _collection_shard2_replica1", [junit4:junit4] 1> "numShards":"2", [junit4:junit4] 1> "shard":"shard2", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"recovering", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"nodes_used_collection_shard2_replica1", [junit4:junit4] 1> "collection":"nodes_used_collection", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:49741__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:49741/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.801; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D2 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":"127.0.0.1:60642__r%2Feq_nodes_used= _collection_shard1_replica2", [junit4:junit4] 1> "numShards":"2", [junit4:junit4] 1> "shard":"shard1", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"recovering", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"nodes_used_collection_shard1_replica2", [junit4:junit4] 1> "collection":"nodes_used_collection", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:60642__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:60642/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.804; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D1 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":null, [junit4:junit4] 1> "numShards":"1", [junit4:junit4] 1> "shard":"shard1", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"down", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"awholynewcollection_0_shard1_replica2", [junit4:junit4] 1> "collection":"awholynewcollection_0", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:51825__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:51825/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.804; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Create collection awholynewcollection_0 with= numShards 1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.806; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D1 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":null, [junit4:junit4] 1> "numShards":"1", [junit4:junit4] 1> "shard":"shard1", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"down", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"awholynewcollection_0_shard1_replica5", [junit4:junit4] 1> "collection":"awholynewcollection_0", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:60700__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:60700/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.809; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D1 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":null, [junit4:junit4] 1> "numShards":"1", [junit4:junit4] 1> "shard":"shard1", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"down", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"awholynewcollection_0_shard1_replica3", [junit4:junit4] 1> "collection":"awholynewcollection_0", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:49741__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:49741/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.813; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D1 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":null, [junit4:junit4] 1> "numShards":"1", [junit4:junit4] 1> "shard":"shard1", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"down", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"awholynewcollection_0_shard1_replica1", [junit4:junit4] 1> "collection":"awholynewcollection_0", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:58902__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:58902/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.816; org.apache.solr.cloud= .Overseer$ClusterStateUpdater; Update state numShards=3D1 message=3D{ [junit4:junit4] 1> "operation":"state", [junit4:junit4] 1> "core_node_name":null, [junit4:junit4] 1> "numShards":"1", [junit4:junit4] 1> "shard":"shard1", [junit4:junit4] 1> "roles":null, [junit4:junit4] 1> "state":"down", [junit4:junit4] 1> "shard_state":"active", [junit4:junit4] 1> "core":"awholynewcollection_0_shard1_replica4", [junit4:junit4] 1> "collection":"awholynewcollection_0", [junit4:junit4] 1> "shard_range":null, [junit4:junit4] 1> "node_name":"127.0.0.1:60642__r%2Feq", [junit4:junit4] 1> "base_url":"http://127.0.0.1:60642/_r/eq"} [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.819; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.820; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.820; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.819; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.819; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.820; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.820; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:50.820; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.648; org.apache.solr.handl= er.admin.CoreAdminHandler; Waited coreNodeName: 127.0.0.1:49741__r%2Feq_nod= es_used_collection_shard2_replica1, state: recovering, checkLive: true, onl= yIfLeader: true for: 2 seconds. [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.648; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= coreNodeName=3D127.0.0.1:49741__r%252Feq_nodes_used_collection_shard2_repli= ca1&state=3Drecovering&nodeName=3D127.0.0.1:49741__r%252Feq&action=3DPREPRE= COVERY&checkLive=3Dtrue&core=3Dnodes_used_collection_shard2_replica2&wt=3Dj= avabin&onlyIfLeader=3Dtrue&version=3D2} status=3D0 QTime=3D2001=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.694; org.apache.solr.core.= CoreContainer; registering core: awholynewcollection_0_shard1_replica2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.694; org.apache.solr.core.= CoreContainer; registering core: awholynewcollection_0_shard1_replica5 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.695; org.apache.solr.cloud= .ZkController; Register replica - core:awholynewcollection_0_shard1_replica= 5 address:http://127.0.0.1:60700/_r/eq collection:awholynewcollection_0 sha= rd:shard1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.695; org.apache.solr.cloud= .ZkController; Register replica - core:awholynewcollection_0_shard1_replica= 2 address:http://127.0.0.1:51825/_r/eq collection:awholynewcollection_0 sha= rd:shard1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.696; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0/leader_e= lect/shard1/election [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.696; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0/leader_e= lect/shard1/election [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.701; org.apache.solr.cloud= .ShardLeaderElectionContext; Running the leader process. [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.703; org.apache.solr.handl= er.admin.CoreAdminHandler; Waited coreNodeName: 127.0.0.1:60642__r%2Feq_nod= es_used_collection_shard1_replica2, state: recovering, checkLive: true, onl= yIfLeader: true for: 2 seconds. [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.703; org.apache.solr.cloud= .ShardLeaderElectionContext; Waiting until we see more replicas up: total= =3D5 found=3D2 timeoutin=3D180000 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.703; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= coreNodeName=3D127.0.0.1:60642__r%252Feq_nodes_used_collection_shard1_repli= ca2&state=3Drecovering&nodeName=3D127.0.0.1:60642__r%252Feq&action=3DPREPRE= COVERY&checkLive=3Dtrue&core=3Dnodes_used_collection_shard1_replica1&wt=3Dj= avabin&onlyIfLeader=3Dtrue&version=3D2} status=3D0 QTime=3D2001=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.735; org.apache.solr.core.= CoreContainer; registering core: awholynewcollection_0_shard1_replica3 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.735; org.apache.solr.cloud= .ZkController; Register replica - core:awholynewcollection_0_shard1_replica= 3 address:http://127.0.0.1:49741/_r/eq collection:awholynewcollection_0 sha= rd:shard1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.792; org.apache.solr.core.= CoreContainer; registering core: awholynewcollection_0_shard1_replica1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.793; org.apache.solr.cloud= .ZkController; Register replica - core:awholynewcollection_0_shard1_replica= 1 address:http://127.0.0.1:58902/_r/eq collection:awholynewcollection_0 sha= rd:shard1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.793; org.apache.solr.core.= CoreContainer; registering core: awholynewcollection_0_shard1_replica4 [junit4:junit4] 1> INFO - 2013-04-17 08:10:51.793; org.apache.solr.cloud= .ZkController; Register replica - core:awholynewcollection_0_shard1_replica= 4 address:http://127.0.0.1:60642/_r/eq collection:awholynewcollection_0 sha= rd:shard1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.204; org.apache.solr.cloud= .ShardLeaderElectionContext; Enough replicas found to continue. [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.205; org.apache.solr.cloud= .ShardLeaderElectionContext; I may be the new leader - try and sync [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.205; org.apache.solr.cloud= .SyncStrategy; Sync replicas to http://127.0.0.1:60700/_r/eq/awholynewcolle= ction_0_shard1_replica5/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.205; org.apache.solr.updat= e.PeerSync; PeerSync: core=3Dawholynewcollection_0_shard1_replica5 url=3Dht= tp://127.0.0.1:60700/_r/eq START replicas=3D[http://127.0.0.1:51825/_r/eq/a= wholynewcollection_0_shard1_replica2/, http://127.0.0.1:49741/_r/eq/awholyn= ewcollection_0_shard1_replica3/, http://127.0.0.1:58902/_r/eq/awholynewcoll= ection_0_shard1_replica1/, http://127.0.0.1:60642/_r/eq/awholynewcollection= _0_shard1_replica4/] nUpdates=3D100 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.206; org.apache.solr.updat= e.PeerSync; PeerSync: core=3Dawholynewcollection_0_shard1_replica5 url=3Dht= tp://127.0.0.1:60700/_r/eq DONE. We have no versions. sync failed. [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.207; org.apache.solr.cloud= .SyncStrategy; Leader's attempt to sync with shard failed, moving to the ne= xt canidate [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.208; org.apache.solr.cloud= .ShardLeaderElectionContext; We failed sync, but we have no versions - we c= an't sync in that case - we were active before, so become leader anyway [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.208; org.apache.solr.cloud= .ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:60700/_r= /eq/awholynewcollection_0_shard1_replica5/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.208; org.apache.solr.commo= n.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0/leaders/= shard1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.211; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica3] webapp=3D/_r/eq path=3D/g= et params=3D{getVersions=3D100&distrib=3Dfalse&wt=3Djavabin&qt=3D/get&versi= on=3D2} status=3D0 QTime=3D0=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.211; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica1] webapp=3D/_r/eq path=3D/g= et params=3D{getVersions=3D100&distrib=3Dfalse&wt=3Djavabin&qt=3D/get&versi= on=3D2} status=3D0 QTime=3D0=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.215; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica4] webapp=3D/_r/eq path=3D/g= et params=3D{getVersions=3D100&distrib=3Dfalse&wt=3Djavabin&qt=3D/get&versi= on=3D2} status=3D0 QTime=3D0=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.215; org.apache.solr.core.= SolrCore; [awholynewcollection_0_shard1_replica2] webapp=3D/_r/eq path=3D/g= et params=3D{getVersions=3D100&distrib=3Dfalse&wt=3Djavabin&qt=3D/get&versi= on=3D2} status=3D0 QTime=3D3=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.334; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.402; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.402; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.402; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.402; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.402; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.402; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.402; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.402; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeDataChanged path:/clusterstate.json, has occurred - updatin= g... (live nodes size: 5) [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.419; org.apache.solr.cloud= .ZkController; We are http://127.0.0.1:60700/_r/eq/awholynewcollection_0_sh= ard1_replica5/ and leader is http://127.0.0.1:60700/_r/eq/awholynewcollecti= on_0_shard1_replica5/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.420; org.apache.solr.cloud= .ZkController; No LogReplay needed for core=3Dawholynewcollection_0_shard1_= replica5 baseURL=3Dhttp://127.0.0.1:60700/_r/eq [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.420; org.apache.solr.cloud= .ZkController; I am the leader, no recovery necessary [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.421; org.apache.solr.cloud= .ZkController; publishing core=3Dawholynewcollection_0_shard1_replica5 stat= e=3Dactive [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.422; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.423; org.apache.solr.core.= CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene= -Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Collec= tionsAPIDistributedZkTest-jetty1-1366186241726/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.424; org.apache.solr.core.= SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lu= cene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Co= llectionsAPIDistributedZkTest-jetty1-1366186241726/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.425; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= numShards=3D1&shard=3Dshard1&name=3Dawholynewcollection_0_shard1_replica5&a= ction=3DCREATE&collection=3Dawholynewcollection_0&wt=3Djavabin&qt=3D/admin/= cores&version=3D2} status=3D0 QTime=3D2641=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.702; org.apache.solr.cloud= .ZkController; We are http://127.0.0.1:51825/_r/eq/awholynewcollection_0_sh= ard1_replica2/ and leader is http://127.0.0.1:60700/_r/eq/awholynewcollecti= on_0_shard1_replica5/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.703; org.apache.solr.cloud= .ZkController; No LogReplay needed for core=3Dawholynewcollection_0_shard1_= replica2 baseURL=3Dhttp://127.0.0.1:51825/_r/eq [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.703; org.apache.solr.cloud= .ZkController; Core needs to recover:awholynewcollection_0_shard1_replica2 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.703; org.apache.solr.updat= e.DefaultSolrCoreState; Running recovery - first canceling any ongoing reco= very [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.704; org.apache.solr.cloud= .RecoveryStrategy; Starting recovery process. core=3Dawholynewcollection_0= _shard1_replica2 recoveringAfterStartup=3Dtrue [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.704; org.apache.solr.cloud= .RecoveryStrategy; ###### startupVersions=3D[] [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.704; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.704; org.apache.solr.cloud= .ZkController; publishing core=3Dawholynewcollection_0_shard1_replica2 stat= e=3Drecovering [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.705; org.apache.solr.core.= CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene= -Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Collec= tionsAPIDistributedZkTest-jetty2-1366186242905/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.706; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.706; org.apache.solr.core.= SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lu= cene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Co= llectionsAPIDistributedZkTest-jetty2-1366186242905/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.707; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= numShards=3D1&shard=3Dshard1&name=3Dawholynewcollection_0_shard1_replica2&a= ction=3DCREATE&collection=3Dawholynewcollection_0&wt=3Djavabin&qt=3D/admin/= cores&version=3D2} status=3D0 QTime=3D2924=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.710; org.apache.solr.handl= er.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:51825_= _r%2Feq_awholynewcollection_0_shard1_replica2, state: recovering, checkLive= : true, onlyIfLeader: true [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.739; org.apache.solr.cloud= .ZkController; We are http://127.0.0.1:49741/_r/eq/awholynewcollection_0_sh= ard1_replica3/ and leader is http://127.0.0.1:60700/_r/eq/awholynewcollecti= on_0_shard1_replica5/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.739; org.apache.solr.cloud= .ZkController; No LogReplay needed for core=3Dawholynewcollection_0_shard1_= replica3 baseURL=3Dhttp://127.0.0.1:49741/_r/eq [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.739; org.apache.solr.cloud= .ZkController; Core needs to recover:awholynewcollection_0_shard1_replica3 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.740; org.apache.solr.updat= e.DefaultSolrCoreState; Running recovery - first canceling any ongoing reco= very [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.740; org.apache.solr.cloud= .RecoveryStrategy; Starting recovery process. core=3Dawholynewcollection_0= _shard1_replica3 recoveringAfterStartup=3Dtrue [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.740; org.apache.solr.cloud= .RecoveryStrategy; ###### startupVersions=3D[] [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.740; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.741; org.apache.solr.cloud= .ZkController; publishing core=3Dawholynewcollection_0_shard1_replica3 stat= e=3Drecovering [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.741; org.apache.solr.core.= CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene= -Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Collec= tionsAPIDistributedZkTest-jetty4-1366186245364/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.742; org.apache.solr.core.= SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lu= cene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Co= llectionsAPIDistributedZkTest-jetty4-1366186245364/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.742; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.743; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= numShards=3D1&shard=3Dshard1&name=3Dawholynewcollection_0_shard1_replica3&a= ction=3DCREATE&collection=3Dawholynewcollection_0&wt=3Djavabin&qt=3D/admin/= cores&version=3D2} status=3D0 QTime=3D2960=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.746; org.apache.solr.handl= er.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:49741_= _r%2Feq_awholynewcollection_0_shard1_replica3, state: recovering, checkLive= : true, onlyIfLeader: true [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.844; org.apache.solr.cloud= .ZkController; We are http://127.0.0.1:58902/_r/eq/awholynewcollection_0_sh= ard1_replica1/ and leader is http://127.0.0.1:60700/_r/eq/awholynewcollecti= on_0_shard1_replica5/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.845; org.apache.solr.cloud= .ZkController; No LogReplay needed for core=3Dawholynewcollection_0_shard1_= replica1 baseURL=3Dhttp://127.0.0.1:58902/_r/eq [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.845; org.apache.solr.cloud= .ZkController; Core needs to recover:awholynewcollection_0_shard1_replica1 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.845; org.apache.solr.updat= e.DefaultSolrCoreState; Running recovery - first canceling any ongoing reco= very [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.846; org.apache.solr.cloud= .ZkController; We are http://127.0.0.1:60642/_r/eq/awholynewcollection_0_sh= ard1_replica4/ and leader is http://127.0.0.1:60700/_r/eq/awholynewcollecti= on_0_shard1_replica5/ [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.846; org.apache.solr.cloud= .RecoveryStrategy; Starting recovery process. core=3Dawholynewcollection_0= _shard1_replica1 recoveringAfterStartup=3Dtrue [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.846; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.846; org.apache.solr.cloud= .ZkController; No LogReplay needed for core=3Dawholynewcollection_0_shard1_= replica4 baseURL=3Dhttp://127.0.0.1:60642/_r/eq [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.846; org.apache.solr.cloud= .RecoveryStrategy; ###### startupVersions=3D[] [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.847; org.apache.solr.core.= CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene= -Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Collec= tionsAPIDistributedZkTest-controljetty-1366186241479/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.847; org.apache.solr.cloud= .ZkController; Core needs to recover:awholynewcollection_0_shard1_replica4 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.848; org.apache.solr.updat= e.DefaultSolrCoreState; Running recovery - first canceling any ongoing reco= very [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.848; org.apache.solr.core.= SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lu= cene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Co= llectionsAPIDistributedZkTest-controljetty-1366186241479/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.847; org.apache.solr.cloud= .ZkController; publishing core=3Dawholynewcollection_0_shard1_replica1 stat= e=3Drecovering [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.849; org.apache.solr.commo= n.cloud.ZkStateReader; Updating cloud state from ZooKeeper...=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.849; org.apache.solr.cloud= .RecoveryStrategy; Starting recovery process. core=3Dawholynewcollection_0= _shard1_replica4 recoveringAfterStartup=3Dtrue [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.849; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= numShards=3D1&shard=3Dshard1&name=3Dawholynewcollection_0_shard1_replica1&a= ction=3DCREATE&collection=3Dawholynewcollection_0&wt=3Djavabin&qt=3D/admin/= cores&version=3D2} status=3D0 QTime=3D3066=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.850; org.apache.solr.cloud= .RecoveryStrategy; ###### startupVersions=3D[] [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.850; org.apache.solr.cloud= .ZkController; publishing core=3Dawholynewcollection_0_shard1_replica4 stat= e=3Drecovering [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.850; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.850; org.apache.solr.core.= CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene= -Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Collec= tionsAPIDistributedZkTest-jetty3-1366186244083/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.851; org.apache.solr.core.= SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lu= cene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.Co= llectionsAPIDistributedZkTest-jetty3-1366186244083/solr.xml [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.851; org.apache.solr.clien= t.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnection= s=3D128&maxConnectionsPerHost=3D32&followRedirects=3Dfalse [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.855; org.apache.solr.servl= et.SolrDispatchFilter; [admin] webapp=3Dnull path=3D/admin/cores params=3D{= numShards=3D1&shard=3Dshard1&name=3Dawholynewcollection_0_shard1_replica4&a= ction=3DCREATE&collection=3Dawholynewcollection_0&wt=3Djavabin&qt=3D/admin/= cores&version=3D2} status=3D0 QTime=3D3072=20 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.855; org.apache.solr.cloud= .OverseerCollectionProcessor; Finished create command on all shards for col= lection: awholynewcollection_0 [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.857; org.apache.solr.handl= er.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:58902_= _r%2Feq_awholynewcollection_0_shard1_replica1, state: recovering, checkLive= : true, onlyIfLeader: true [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.858; org.apache.solr.handl= er.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:60642_= _r%2Feq_awholynewcollection_0_shard1_replica4, state: recovering, checkLive= : true, onlyIfLeader: true [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.882; org.apache.solr.cloud= .DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/colle= ction-queue-work/qnr-0000000002 state: SyncConnected type NodeDataChanged [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.883; org.apache.solr.cloud= .DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/colle= ction-queue-work state: SyncConnected type NodeChildrenChanged [junit4:junit4] 1> INFO - 2013-04-17 08:10:52.883; org.apache.solr.cloud= .OverseerCollectionProcessor; Overseer Collection Processor: Message id:/ov= erseer/collection-queue-work/qn-0000000002 complete, response:{success=3D{n= ull=3D{responseHeader=3D{status=3D0,QTime=3D2641},core=3Dawholynewcollectio= n_0_shard1_replica5,saved=3D/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linu= x/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistri= butedZkTest-jetty1-1366186241726/solr.xml},null=3D{responseHeader=3D{status= =3D0,QTime=3D2924},core=3Dawholynewcollection_0_shard1_replica2,saved=3D/mn= t/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/= ./org.apache.solr.cloud.CollectionsAPIDistribu [...truncated too long message...] ionsAPIDistributedZkTest-1366186241263 [junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=3DCollectio= nsAPIDistributedZkTest -Dtests.method=3DtestDistribSearch -Dtests.seed=3DDE= E6CB3BD5AFCE84 -Dtests.multiplier=3D3 -Dtests.slow=3Dtrue -Dtests.locale=3D= no -Dtests.timezone=3DAmerica/North_Dakota/Center -Dtests.file.encoding=3DI= SO-8859-1 [junit4:junit4] FAILURE 274s J1 | CollectionsAPIDistributedZkTest.testDist= ribSearch <<< [junit4:junit4] > Throwable #1: java.lang.AssertionError: There are stil= l nodes recoverying - waited for 230 seconds [junit4:junit4] > =09at __randomizedtesting.SeedInfo.seed([DEE6CB3BD5AFC= E84:5F004523A2F0AEB8]:0) [junit4:junit4] > =09at org.apache.solr.cloud.AbstractDistribZkTestBase.= waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173) [junit4:junit4] > =09at org.apache.solr.cloud.AbstractDistribZkTestBase.= waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131) [junit4:junit4] > =09at org.apache.solr.cloud.AbstractDistribZkTestBase.= waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126) [junit4:junit4] > =09at org.apache.solr.cloud.CollectionsAPIDistributedZ= kTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:509) [junit4:junit4] > =09at org.apache.solr.cloud.CollectionsAPIDistributedZ= kTest.doTest(CollectionsAPIDistributedZkTest.java:145) [junit4:junit4] > =09at org.apache.solr.BaseDistributedSearchTestCase.te= stDistribSearch(BaseDistributedSearchTestCase.java:806) [junit4:junit4] > =09at java.lang.Thread.run(Thread.java:722) [junit4:junit4] 1> INFO - 2013-04-17 08:15:15.123; org.apache.solr.SolrT= estCaseJ4; ###deleteCore [junit4:junit4] 2> 273869 T1938 ccr.ThreadLeakControl.checkThreadLeaks WA= RNING Will linger awaiting termination of 2 leaked thread(s). [junit4:junit4] 1> WARN - 2013-04-17 08:15:16.326; org.apache.solr.commo= n.cloud.ZkStateReader$3; ZooKeeper watch triggered, but Solr cannot talk to= ZK [junit4:junit4] 1> INFO - 2013-04-17 08:15:16.327; org.apache.solr.commo= n.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncCon= nected type:NodeChildrenChanged path:/live_nodes, has occurred - updating..= . (live nodes size: 1) [junit4:junit4] 1> WARN - 2013-04-17 08:15:16.327; org.apache.solr.commo= n.cloud.ZkStateReader$2; ZooKeeper watch triggered, but Solr cannot talk to= ZK [junit4:junit4] 1> WARN - 2013-04-17 08:15:16.328; org.apache.solr.cloud= .LeaderElector$1;=20 [junit4:junit4] 1> org.apache.zookeeper.KeeperException$SessionExpiredExc= eption: KeeperErrorCode =3D Session expired for /collections/awholynewcolle= ction_2/leader_elect/shard2/election [junit4:junit4] 1> =09at org.apache.zookeeper.KeeperException.create(Keep= erException.java:127) [junit4:junit4] 1> =09at org.apache.zookeeper.KeeperException.create(Keep= erException.java:51) [junit4:junit4] 1> =09at org.apache.zookeeper.ZooKeeper.getChildren(ZooKe= eper.java:1468) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.SolrZkClient$6.exec= ute(SolrZkClient.java:235) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.SolrZkClient$6.exec= ute(SolrZkClient.java:232) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.ZkCmdExecutor.retry= Operation(ZkCmdExecutor.java:65) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.SolrZkClient.getChi= ldren(SolrZkClient.java:232) [junit4:junit4] 1> =09at org.apache.solr.cloud.LeaderElector.checkIfIamLe= ader(LeaderElector.java:84) [junit4:junit4] 1> =09at org.apache.solr.cloud.LeaderElector.access$000(L= eaderElector.java:55) [junit4:junit4] 1> =09at org.apache.solr.cloud.LeaderElector$1.process(Le= aderElector.java:129) [junit4:junit4] 1> =09at org.apache.zookeeper.ClientCnxn$EventThread.proc= essEvent(ClientCnxn.java:519) [junit4:junit4] 1> =09at org.apache.zookeeper.ClientCnxn$EventThread.run(= ClientCnxn.java:495) [junit4:junit4] 1> WARN - 2013-04-17 08:15:16.328; org.apache.solr.cloud= .LeaderElector$1;=20 [junit4:junit4] 1> org.apache.zookeeper.KeeperException$SessionExpiredExc= eption: KeeperErrorCode =3D Session expired for /collections/awholynewcolle= ction_2/leader_elect/shard3/election [junit4:junit4] 1> =09at org.apache.zookeeper.KeeperException.create(Keep= erException.java:127) [junit4:junit4] 1> =09at org.apache.zookeeper.KeeperException.create(Keep= erException.java:51) [junit4:junit4] 1> =09at org.apache.zookeeper.ZooKeeper.getChildren(ZooKe= eper.java:1468) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.SolrZkClient$6.exec= ute(SolrZkClient.java:235) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.SolrZkClient$6.exec= ute(SolrZkClient.java:232) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.ZkCmdExecutor.retry= Operation(ZkCmdExecutor.java:65) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.SolrZkClient.getChi= ldren(SolrZkClient.java:232) [junit4:junit4] 1> =09at org.apache.solr.cloud.LeaderElector.checkIfIamLe= ader(LeaderElector.java:84) [junit4:junit4] 1> =09at org.apache.solr.cloud.LeaderElector.access$000(L= eaderElector.java:55) [junit4:junit4] 1> =09at org.apache.solr.cloud.LeaderElector$1.process(Le= aderElector.java:129) [junit4:junit4] 1> =09at org.apache.zookeeper.ClientCnxn$EventThread.proc= essEvent(ClientCnxn.java:519) [junit4:junit4] 1> =09at org.apache.zookeeper.ClientCnxn$EventThread.run(= ClientCnxn.java:495) [junit4:junit4] 1> WARN - 2013-04-17 08:15:16.329; org.apache.solr.cloud= .LeaderElector$1;=20 [junit4:junit4] 1> org.apache.zookeeper.KeeperException$SessionExpiredExc= eption: KeeperErrorCode =3D Session expired for /collections/awholynewcolle= ction_2/leader_elect/shard1/election [junit4:junit4] 1> =09at org.apache.zookeeper.KeeperException.create(Keep= erException.java:127) [junit4:junit4] 1> =09at org.apache.zookeeper.KeeperException.create(Keep= erException.java:51) [junit4:junit4] 1> =09at org.apache.zookeeper.ZooKeeper.getChildren(ZooKe= eper.java:1468) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.SolrZkClient$6.exec= ute(SolrZkClient.java:235) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.SolrZkClient$6.exec= ute(SolrZkClient.java:232) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.ZkCmdExecutor.retry= Operation(ZkCmdExecutor.java:65) [junit4:junit4] 1> =09at org.apache.solr.common.cloud.SolrZkClient.getChi= ldren(SolrZkClient.java:232) [junit4:junit4] 1> =09at org.apache.solr.cloud.LeaderElector.checkIfIamLe= ader(LeaderElector.java:84) [junit4:junit4] 1> =09at org.apache.solr.cloud.LeaderElector.access$000(L= eaderElector.java:55) [junit4:junit4] 1> =09at org.apache.solr.cloud.LeaderElector$1.process(Le= aderElector.java:129) [junit4:junit4] 1> =09at org.apache.zookeeper.ClientCnxn$EventThread.proc= essEvent(ClientCnxn.java:519) [junit4:junit4] 1> =09at org.apache.zookeeper.ClientCnxn$EventThread.run(= ClientCnxn.java:495) [junit4:junit4] 2> NOTE: test params are: codec=3DAsserting, sim=3DDefaul= tSimilarity, locale=3Dno, timezone=3DAmerica/North_Dakota/Center [junit4:junit4] 2> NOTE: Linux 3.2.0-40-generic i386/Oracle Corporation 1= .7.0_17 (32-bit)/cpus=3D8,threads=3D1,free=3D16850416,total=3D122523648 [junit4:junit4] 2> NOTE: All tests run in this JVM: [TestPostingsSolrHigh= lighter, TestDistributedSearch, PathHierarchyTokenizerFactoryTest, TestSolr= QueryParserResource, AliasIntegrationTest, TestArbitraryIndexDir, Suggester= WFSTTest, TestFieldCollectionResource, SolrCmdDistributorTest, CopyFieldTes= t, HighlighterConfigTest, ZkNodePropsTest, ClusterStateTest, TestFiltering,= BadCopyFieldTest, TestFoldingMultitermQuery, TestDynamicFieldResource, Ful= lSolrCloudDistribCmdsTest, IndexSchemaRuntimeFieldTest, TestSolrXMLSerializ= er, TermVectorComponentDistributedTest, SignatureUpdateProcessorFactoryTest= , DisMaxRequestHandlerTest, TestPropInject, TestNumberUtils, TestSuggestSpe= llingConverter, DirectUpdateHandlerTest, IndexBasedSpellCheckerTest, TestLM= JelinekMercerSimilarityFactory, UpdateParamsTest, TestCollationKeyRangeQuer= ies, TestBinaryResponseWriter, TestCollationField, SampleTest, TestFastOutp= utStream, TestStressLucene, SimplePostToolTest, ZkSolrClientTest, StatsComp= onentTest, TestComponentsName, FileUtilsTest, TestFieldTypeResource, TestWo= rdDelimiterFilterFactory, StandardRequestHandlerTest, SpellPossibilityItera= torTest, HighlighterTest, TestSolrQueryParserDefaultOperatorResource, Outpu= tWriterTest, SolrIndexConfigTest, DebugComponentTest, TestJmxIntegration, D= ateMathParserTest, IndexReaderFactoryTest, ConvertedLegacyTest, TestIndexSe= archer, TestSchemaSimilarityResource, TestFieldResource, TestSolr4Spatial, = NotRequiredUniqueKeyTest, SolrCoreCheckLockOnStartupTest, TestUpdate, Sugge= sterTSTTest, CurrencyFieldXmlFileTest, OpenCloseCoreStressTest, TestRTGBase= , TestQuerySenderNoQuery, SpellCheckCollatorTest, TestZkChroot, ShardRoutin= gCustomTest, DirectSolrSpellCheckerTest, ChaosMonkeySafeLeaderTest, BasicDi= stributedZk2Test, CollectionsAPIDistributedZkTest] [junit4:junit4] Completed on J1 in 275.15s, 1 test, 1 failure <<< FAILURES! [...truncated 439 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:381: The followi= ng error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:361: The followi= ng error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The followin= g error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:183: The fo= llowing error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:438:= The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:12= 34: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:87= 8: There were test failures: 291 suites, 1209 tests, 1 failure, 13 ignored = (7 assumptions) Total time: 39 minutes 26 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Description set: Java: 32bit/jdk1.7.0_17 -server -XX:+UseSerialGC Email was triggered for: Failure Sending email for trigger: Failure ------=_Part_14_1011344083.1366186951786 Content-Type: text/plain; charset=us-ascii --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org ------=_Part_14_1011344083.1366186951786--