lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1992 - Still Failing!
Date Sat, 13 Dec 2014 11:11:28 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1992/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC (asserts: true)

1 tests failed.
FAILED:  org.apache.solr.cloud.DeleteReplicaTest.testDistribSearch

Error Message:
Should have had a good message here

Stack Trace:
java.lang.AssertionError: Should have had a good message here
	at __randomizedtesting.SeedInfo.seed([D2DD35915246F37B:533BBB8925199347]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:138)
	at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:89)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 9133 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/init-core-data-001
   [junit4]   2> 2349462 T7794 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 2349463 T7794 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /u_ksd/q
   [junit4]   2> 2349471 T7794 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 2349474 T7794 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2349476 T7795 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2349578 T7794 oasc.ZkTestServer.run start zk server on port:51526
   [junit4]   2> 2349580 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2349583 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2349597 T7802 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5db7158e name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2349598 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2349598 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2349599 T7794 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2349608 T7796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14a4308b7540000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2349611 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2349612 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2349616 T7805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63d3c48f name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2349616 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2349617 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2349617 T7794 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 2349623 T7794 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 2349629 T7794 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 2349635 T7794 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 2349641 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2349642 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2349652 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2349653 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 2349660 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2349661 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2349666 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2349667 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2349673 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2349673 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2349679 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2349680 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 2349687 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2349688 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2349696 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2349697 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2349703 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2349704 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2349712 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2349712 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2349720 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2349721 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2351605 T7794 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2351643 T7794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51529
   [junit4]   2> 2351644 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2351644 T7794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2351644 T7794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002
   [junit4]   2> 2351645 T7794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/'
   [junit4]   2> 2351692 T7794 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/solr.xml
   [junit4]   2> 2351716 T7794 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002
   [junit4]   2> 2351717 T7794 oasc.CoreContainer.<init> New CoreContainer 1704627795
   [junit4]   2> 2351717 T7794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/]
   [junit4]   2> 2351718 T7794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2351719 T7794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2351720 T7794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2351720 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2351721 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 2351721 T7794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2351722 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2351722 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2351723 T7794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2351723 T7794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2351724 T7794 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2351725 T7794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2351728 T7794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2351728 T7794 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2351729 T7794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51526/solr
   [junit4]   2> 2351730 T7794 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2351730 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2351733 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2351739 T7817 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4355027c name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2351739 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2351739 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2351751 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2351766 T7820 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62686eca name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2351767 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2351772 T7794 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2351781 T7794 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2351790 T7794 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2351800 T7794 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2351811 T7794 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2351821 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2351827 T7794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51529_u_ksd%2Fq
   [junit4]   2> 2351831 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51529_u_ksd%2Fq
   [junit4]   2> 2351839 T7794 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2351846 T7794 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2351852 T7794 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2351858 T7794 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:51529_u_ksd%2Fq
   [junit4]   2> 2351859 T7794 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2351865 T7794 oasc.Overseer.start Overseer (id=92960447029051395-127.0.0.1:51529_u_ksd%2Fq-n_0000000000) starting
   [junit4]   2> 2351873 T7794 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2351902 T7794 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2351904 T7822 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2351908 T7794 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2351914 T7794 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2351919 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2351929 T7821 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2351932 T7794 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002
   [junit4]   2> 2352803 T7794 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/conf/
   [junit4]   2> 2352922 T7794 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 2352925 T7794 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 2352925 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2352926 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2352927 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2352935 T7827 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@96dddff name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2352936 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2352936 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2352940 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2352952 T7794 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2352953 T7794 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
   [junit4]   2> 2352953 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2352955 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2352964 T7830 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@da60823 name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2352964 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2352964 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2352971 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2352973 T7796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14a4308b7540005, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2353011 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 0, message = {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "name":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "stateFormat":2} current state version: 0
   [junit4]   2> 2353012 T7821 oasco.ClusterStateMutator.createCollection building a new cName: collection1
   [junit4]   2> 2353013 T7821 oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/collection1/state.json
   [junit4]   2> 2353019 T7820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2353019 T7827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2354899 T7794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 2354900 T7794 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2354908 T7794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51534
   [junit4]   2> 2354910 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2354910 T7794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2354911 T7794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003
   [junit4]   2> 2354911 T7794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003/'
   [junit4]   2> 2354955 T7794 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003/solr.xml
   [junit4]   2> 2354968 T7794 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003
   [junit4]   2> 2354969 T7794 oasc.CoreContainer.<init> New CoreContainer 1474863892
   [junit4]   2> 2354969 T7794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003/]
   [junit4]   2> 2354970 T7794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2354970 T7794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2354971 T7794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2354971 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2354971 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 2354972 T7794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2354972 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2354972 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2354973 T7794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2354973 T7794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2354973 T7794 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2354974 T7794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2354975 T7794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2354975 T7794 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2354975 T7794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51526/solr
   [junit4]   2> 2354975 T7794 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2354976 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2354978 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2354983 T7842 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40414610 name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2354983 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2354984 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2354991 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2354996 T7845 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45351664 name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2354997 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2355016 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2356034 T7794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51534_u_ksd%2Fq
   [junit4]   2> 2356039 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51534_u_ksd%2Fq
   [junit4]   2> 2356057 T7794 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2356064 T7794 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003
   [junit4]   2> 2356701 T7794 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003/conf/
   [junit4]   2> 2356806 T7794 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 2356809 T7794 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 2356810 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2358880 T7794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 2358880 T7794 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2358888 T7794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51537
   [junit4]   2> 2358892 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2358893 T7794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2358894 T7794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004
   [junit4]   2> 2358895 T7794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004/'
   [junit4]   2> 2358938 T7794 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004/solr.xml
   [junit4]   2> 2358955 T7794 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004
   [junit4]   2> 2358955 T7794 oasc.CoreContainer.<init> New CoreContainer 1886158987
   [junit4]   2> 2358956 T7794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004/]
   [junit4]   2> 2358956 T7794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2358957 T7794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2358958 T7794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2358959 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2358959 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 2358959 T7794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2358960 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2358960 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2358961 T7794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2358961 T7794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2358962 T7794 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2358963 T7794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2358963 T7794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2358963 T7794 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2358964 T7794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51526/solr
   [junit4]   2> 2358964 T7794 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2358964 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2358966 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2358971 T7858 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16529c7a name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2358971 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2358972 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2358985 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2358989 T7861 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ff258ba name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2358990 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2359065 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2360080 T7794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51537_u_ksd%2Fq
   [junit4]   2> 2360082 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51537_u_ksd%2Fq
   [junit4]   2> 2360094 T7794 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2360103 T7794 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004
   [junit4]   2> 2360873 T7794 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004/conf/
   [junit4]   2> 2360987 T7794 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 2360990 T7794 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 2360991 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2362993 T7794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 2362995 T7794 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2363007 T7794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51540
   [junit4]   2> 2363009 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2363011 T7794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2363012 T7794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005
   [junit4]   2> 2363012 T7794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/'
   [junit4]   2> 2363066 T7794 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/solr.xml
   [junit4]   2> 2363078 T7794 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005
   [junit4]   2> 2363079 T7794 oasc.CoreContainer.<init> New CoreContainer 1251588988
   [junit4]   2> 2363079 T7794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/]
   [junit4]   2> 2363080 T7794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2363080 T7794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2363081 T7794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2363081 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2363082 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 2363082 T7794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2363082 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2363082 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2363083 T7794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2363084 T7794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2363084 T7794 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2363085 T7794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2363085 T7794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2363086 T7794 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2363086 T7794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51526/solr
   [junit4]   2> 2363086 T7794 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2363087 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2363088 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2363095 T7874 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f519a4f name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2363096 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2363096 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2363103 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2363112 T7877 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10e4893c name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2363112 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2363147 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2364169 T7794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51540_u_ksd%2Fq
   [junit4]   2> 2364173 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51540_u_ksd%2Fq
   [junit4]   2> 2364193 T7794 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2364203 T7794 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005
   [junit4]   2> 2365068 T7794 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/conf/
   [junit4]   2> 2365190 T7794 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 2365192 T7794 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 2365193 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2367105 T7794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 2367107 T7794 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2367114 T7794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51543
   [junit4]   2> 2367116 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2367116 T7794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2367117 T7794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006
   [junit4]   2> 2367117 T7794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006/'
   [junit4]   2> 2367166 T7794 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006/solr.xml
   [junit4]   2> 2367179 T7794 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006
   [junit4]   2> 2367180 T7794 oasc.CoreContainer.<init> New CoreContainer 1777461615
   [junit4]   2> 2367180 T7794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006/]
   [junit4]   2> 2367181 T7794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2367181 T7794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2367181 T7794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2367182 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2367182 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 2367183 T7794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2367183 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2367183 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2367184 T7794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2367184 T7794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2367185 T7794 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2367185 T7794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2367186 T7794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2367186 T7794 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2367187 T7794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51526/solr
   [junit4]   2> 2367187 T7794 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2367187 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2367191 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2367194 T7890 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b051b46 name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2367194 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2367194 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2367201 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2367205 T7893 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@679495f2 name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2367206 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2367234 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2368260 T7794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51543_u_ksd%2Fq
   [junit4]   2> 2368263 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51543_u_ksd%2Fq
   [junit4]   2> 2368281 T7794 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2368805 T7794 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006
   [junit4]   2> 2369621 T7794 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006/conf/
   [junit4]   2> 2369742 T7794 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 2369745 T7794 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 2369745 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2369750 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2369751 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2369756 T7897 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b82547b name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2369756 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2369756 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2369760 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2369771 T7794 oasc.AbstractFullDistribZkTestBase.createCollection Creating collection with stateFormat=2: delLiveColl
   [junit4]   2> 2369771 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2369773 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2369777 T7900 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f250410 name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2369777 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2369777 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2369781 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2369804 T7882 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&maxShardsPerNode=2&name=delLiveColl&action=CREATE&numShards=2&stateFormat=2&wt=javabin&version=2
   [junit4]   2> 2369806 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/collection-queue-work state SyncConnected
   [junit4]   2> 2369808 T7822 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delLiveColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2",
   [junit4]   2> 	  "stateFormat":"2"}
   [junit4]   2> 2369814 T7901 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delLiveColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2",
   [junit4]   2> 	  "stateFormat":"2"}
   [junit4]   2> 2369818 T7901 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
   [junit4]   2> 2369819 T7901 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/delLiveColl 
   [junit4]   2> 2369820 T7901 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
   [junit4]   2> 2369828 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2369831 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delLiveColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2",
   [junit4]   2> 	  "stateFormat":"2"} current state version: 1
   [junit4]   2> 2369831 T7821 oasco.ClusterStateMutator.createCollection building a new cName: delLiveColl
   [junit4]   2> 2369832 T7821 oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/delLiveColl/state.json
   [junit4]   2> 2369840 T7893 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2369840 T7897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2369840 T7900 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2369840 T7877 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2369841 T7820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2369840 T7861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2369840 T7827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2369840 T7845 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2370034 T7901 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection delLiveColl, shardNames [shard1, shard2] , replicationFactor : 2
   [junit4]   2> 2370034 T7901 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica1 as part of slice shard1 of collection delLiveColl on 127.0.0.1:51540_u_ksd%2Fq
   [junit4]   2> 2370036 T7901 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica2 as part of slice shard1 of collection delLiveColl on 127.0.0.1:51529_u_ksd%2Fq
   [junit4]   2> 2370040 T7901 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica1 as part of slice shard2 of collection delLiveColl on 127.0.0.1:51537_u_ksd%2Fq
   [junit4]   2> 2370052 T7871 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard1_replica1&action=CREATE&numShards=2&collection=delLiveColl&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2370056 T7809 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard1_replica2&action=CREATE&numShards=2&collection=delLiveColl&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2370058 T7901 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica2 as part of slice shard2 of collection delLiveColl on 127.0.0.1:51534_u_ksd%2Fq
   [junit4]   2> 2370062 T7871 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica1 state=down collection=delLiveColl
   [junit4]   2> 2370069 T7871 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 2370070 T7871 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 2370070 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2370073 T7853 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard2_replica1&action=CREATE&numShards=2&collection=delLiveColl&shard=shard2&wt=javabin&version=2
   [junit4]   2> 2370075 T7871 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 2370079 T7853 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica1 state=down collection=delLiveColl
   [junit4]   2> 2370085 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"delLiveColl_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:51540/u_ksd/q",
   [junit4]   2> 	  "node_name":"127.0.0.1:51540_u_ksd%2Fq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"} current state version: 2
   [junit4]   2> 2370090 T7834 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard2_replica2&action=CREATE&numShards=2&collection=delLiveColl&shard=shard2&wt=javabin&version=2
   [junit4]   2> 2370092 T7871 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 
   [junit4]   2> 2370092 T7871 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2370096 T7809 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica2 state=down collection=delLiveColl
   [junit4]   2> 2370097 T7821 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delLiveColl_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:51540/u_ksd/q",
   [junit4]   2> 	  "node_name":"127.0.0.1:51540_u_ksd%2Fq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2370098 T7834 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica2 state=down collection=delLiveColl
   [junit4]   2> 2370101 T7853 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 2370101 T7853 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 2370104 T7853 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 2370107 T7834 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 2370108 T7834 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 2370108 T7853 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 
   [junit4]   2> 2370110 T7853 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2370110 T7834 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 2370112 T7809 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 2370113 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 3, message = {
   [junit4]   2> 	  "core":"delLiveColl_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:51537/u_ksd/q",
   [junit4]   2> 	  "node_name":"127.0.0.1:51537_u_ksd%2Fq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"} current state version: 2
   [junit4]   2> 2370115 T7821 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delLiveColl_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:51537/u_ksd/q",
   [junit4]   2> 	  "node_name":"127.0.0.1:51537_u_ksd%2Fq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2370116 T7834 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 
   [junit4]   2> 2370116 T7834 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2370121 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2370123 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 3, message = {
   [junit4]   2> 	  "core":"delLiveColl_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:51534/u_ksd/q",
   [junit4]   2> 	  "node_name":"127.0.0.1:51534_u_ksd%2Fq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"} current state version: 2
   [junit4]   2> 2370125 T7821 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delLiveColl_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:51534/u_ksd/q",
   [junit4]   2> 	  "node_name":"127.0.0.1:51534_u_ksd%2Fq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2370134 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"delLiveColl_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:51529/u_ksd/q",
   [junit4]   2> 	  "node_name":"127.0.0.1:51529_u_ksd%2Fq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"} current state version: 2
   [junit4]   2> 2370135 T7821 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delLiveColl_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:51529/u_ksd/q",
   [junit4]   2> 	  "node_name":"127.0.0.1:51529_u_ksd%2Fq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2370141 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2370144 T7821 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/delLiveColl/state.json version: 0
   [junit4]   2> 2370147 T7845 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2370147 T7861 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2370147 T7877 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2370151 T7861 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 2370151 T7877 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 2370152 T7845 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 2370166 T7809 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 2370168 T7809 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 2370173 T7809 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 2370173 T7809 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2370174 T7809 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica2
   [junit4]   2> 2370174 T7809 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 2370175 T7809 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2370176 T7809 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 2370178 T7809 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2370178 T7809 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/'
   [junit4]   2> 2370262 T7809 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 2370273 T7809 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 2370299 T7809 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 2370320 T7809 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2370324 T7809 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2370333 T7809 oass.IndexSchema.readSchema [delLiveColl_shard1_replica2] Schema name=test
   [junit4]   2> 2370501 T7809 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2370510 T7809 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2370511 T7809 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2370549 T7809 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2370553 T7809 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2370558 T7809 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2370560 T7809 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2370564 T7809 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2370565 T7809 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2370567 T7809 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2370567 T7809 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2370568 T7809 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica2' using configuration from collection delLiveColl
   [junit4]   2> 2370568 T7809 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2370569 T7809 oasc.SolrCore.<init> [delLiveColl_shard1_replica2] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/data/
   [junit4]   2> 2370570 T7809 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b62dc70
   [junit4]   2> 2370571 T7809 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/data
   [junit4]   2> 2370572 T7809 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/data/index/
   [junit4]   2> 2370572 T7809 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica2] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2370572 T7809 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/data/index
   [junit4]   2> 2370573 T7809 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=41, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8412472851871099]
   [junit4]   2> 2370574 T7809 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1625cc00 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@16777a6d),segFN=segments_1,generation=1}
   [junit4]   2> 2370574 T7809 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2370581 T7809 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2370582 T7809 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2370582 T7809 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2370583 T7809 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2370583 T7809 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2370584 T7809 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2370585 T7809 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2370586 T7809 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2370586 T7809 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2370587 T7809 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2370587 T7809 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2370588 T7809 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2370589 T7809 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2370590 T7809 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 2370591 T7809 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
   [junit4]   2> 2370591 T7809 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 2370592 T7809 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 2370592 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 2370593 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 2370594 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 2370594 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 2370595 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 2370596 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 2370596 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 2370597 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 2370597 T7809 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2370598 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2370598 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 2370608 T7809 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2370609 T7809 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2370610 T7809 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2370612 T7809 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2370618 T7809 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2370618 T7809 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2370620 T7809 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2370621 T7809 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1625cc00 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@16777a6d),segFN=segments_1,generation=1}
   [junit4]   2> 2370622 T7809 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2370623 T7809 oass.SolrIndexSearcher.<init> Opening Searcher@580ad4b8[delLiveColl_shard1_replica2] main
   [junit4]   2> 2370623 T7809 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 2370626 T7809 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2370627 T7809 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2370628 T7809 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2370629 T7809 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 2370629 T7809 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2370630 T7809 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2370631 T7809 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2370632 T7809 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 2370638 T7809 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2370639 T7809 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2370639 T7809 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 2370639 T7809 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 2370640 T7809 oasc.CoreContainer.registerCore registering core: delLiveColl_shard1_replica2
   [junit4]   2> 2370640 T7906 oasc.SolrCore.registerSearcher [delLiveColl_shard1_replica2] Registered new searcher Searcher@580ad4b8[delLiveColl_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2370640 T7809 oasc.ZkController.register Register replica - core:delLiveColl_shard1_replica2 address:http://127.0.0.1:51529/u_ksd/q collection:delLiveColl shard:shard1
   [junit4]   2> 2370644 T7809 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl/leader_elect/shard1/election
   [junit4]   2> 2370659 T7809 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 2370663 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 2370665 T7809 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=179998ms
   [junit4]   2> 2370667 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delLiveColl"} current state version: 2
   [junit4]   2> 2370777 T7821 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/delLiveColl/state.json version: 1
   [junit4]   2> 2370781 T7877 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2370781 T7845 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2370781 T7861 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2370783 T7820 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2370785 T7877 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 2370786 T7845 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 2370786 T7861 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 2370793 T7820 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 2371094 T7871 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica1
   [junit4]   2> 2371094 T7871 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 2371095 T7871 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2371096 T7871 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 2371098 T7871 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2371098 T7871 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/'
   [junit4]   2> 2371111 T7853 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica1
   [junit4]   2> 2371112 T7853 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 2371114 T7853 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2371114 T7853 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 2371116 T7853 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2371116 T7853 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004/delLiveColl_shard2_replica1/'
   [junit4]   2> 2371129 T7834 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica2
   [junit4]   2> 2371129 T7834 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 2371129 T7871 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 2371132 T7871 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 2371134 T7834 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2371135 T7834 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 2371142 T7853 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 2371142 T7834 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2371144 T7834 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003/delLiveColl_shard2_replica2/'
   [junit4]   2> 2371156 T7871 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 2371178 T7871 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2371179 T7834 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 2371183 T7853 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 2371190 T7834 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 2371191 T7871 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2371206 T7871 oass.IndexSchema.readSchema [delLiveColl_shard1_replica1] Schema name=test
   [junit4]   2> 2371209 T7834 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 2371200 T7853 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 2371248 T7853 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2371252 T7853 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2371257 T7853 oass.IndexSchema.readSchema [delLiveColl_shard2_replica1] Schema name=test
   [junit4]   2> 2371281 T7834 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2371296 T7834 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2371305 T7834 oass.IndexSchema.readSchema [delLiveColl_shard2_replica2] Schema name=test
   [junit4]   2> 2371529 T7871 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2371533 T7871 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2371535 T7871 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2371549 T7834 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2371550 T7853 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2371556 T7834 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2371557 T7853 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2371557 T7834 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2371561 T7853 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2371584 T7871 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2371593 T7871 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2371598 T7871 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2371600 T7871 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2371601 T7871 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2371601 T7871 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2371604 T7871 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2371604 T7871 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2371605 T7871 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica1' using configuration from collection delLiveColl
   [junit4]   2> 2371605 T7871 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2371605 T7871 oasc.SolrCore.<init> [delLiveColl_shard1_replica1] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/data/
   [junit4]   2> 2371606 T7871 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b62dc70
   [junit4]   2> 2371607 T7871 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/data
   [junit4]   2> 2371607 T7871 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/data/index/
   [junit4]   2> 2371607 T7871 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2371608 T7871 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/data/index
   [junit4]   2> 2371608 T7871 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=41, maxMergeSize=2147483648, maxMergeSi

[...truncated too long message...]

:51540_u_ksd%2Fq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2394368 T7941 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delLiveColl_shard1_replica1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:51540/u_ksd/q",
   [junit4]   2> 	  "node_name":"127.0.0.1:51540_u_ksd%2Fq",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2394378 T7941 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/delLiveColl/state.json version: 22
   [junit4]   2> 2394387 T7941 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2395788 T7926 oascc.ZkStateReader$7.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 2395807 T7794 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/u_ksd/q,null}
   [junit4]   2> 2400792 T7794 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 51543
   [junit4]   2> 2400792 T7794 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1777461615
   [junit4]   2> 2400793 T7794 oasc.Overseer.close Overseer (id=92960447029051405-127.0.0.1:51543_u_ksd%2Fq-n_0000000004) closing
   [junit4]   2> 2400794 T7941 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:51543_u_ksd%2Fq
   [junit4]   2> 2400797 T7940 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 2400797 T7796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14a4308b754000d, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2400813 T7794 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/u_ksd/q,null}
   [junit4]   2> 2405802 T7794 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 2405805 T7794 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51526 51526
   [junit4]   2> 2405902 T7795 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51526 51526
   [junit4]   2> 2406142 T7795 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: 
   [junit4]   2> 	Maximum concurrent create/delete watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		8	/solr/aliases.json
   [junit4]   2> 		8	/solr/clusterstate.json
   [junit4]   2> 		4	/solr/collections/delLiveColl/state.json
   [junit4]   2> 		4	/solr/configs/conf1
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent children watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		8	/solr/live_nodes
   [junit4]   2> 		5	/solr/overseer/queue
   [junit4]   2> 		5	/solr/overseer/collection-queue-work
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=DeleteReplicaTest -Dtests.method=testDistribSearch -Dtests.seed=D2DD35915246F37B -Dtests.slow=true -Dtests.locale=sr_BA_#Latn -Dtests.timezone=Europe/Vienna -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 56.7s | DeleteReplicaTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Should have had a good message here
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D2DD35915246F37B:533BBB8925199347]:0)
   [junit4]    > 	at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:138)
   [junit4]    > 	at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:89)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2406163 T7794 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=sr_BA_#Latn, timezone=Europe/Vienna
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.8.0_20 (64-bit)/cpus=2,threads=1,free=185254816,total=368599040
   [junit4]   2> NOTE: All tests run in this JVM: [TestDefaultSearchFieldResource, TestDistributedGrouping, DistributedFacetPivotSmallTest, TestGroupingSearch, FileUtilsTest, StatelessScriptUpdateProcessorFactoryTest, TestCloudSchemaless, EchoParamsTest, OutputWriterTest, OverseerRolesTest, TestDocSet, TestInitParams, CoreAdminCreateDiscoverTest, AnalysisErrorHandlingTest, TestBM25SimilarityFactory, TestCursorMarkWithoutUniqueKey, QueryParsingTest, TestSolrJ, TestFieldTypeCollectionResource, DistributedFacetPivotLongTailTest, TestManagedResource, JSONWriterTest, LeaderElectionIntegrationTest, UpdateRequestProcessorFactoryTest, TestSolrDeletionPolicy2, TimeZoneUtilsTest, InfoHandlerTest, TestModifyConfFiles, SortByFunctionTest, DocValuesMissingTest, LukeRequestHandlerTest, ConnectionManagerTest, TestComponentsName, HdfsBasicDistributedZk2Test, TestFunctionQuery, ReplicationFactorTest, TestManagedResourceStorage, ResourceLoaderTest, ZkNodePropsTest, CoreContainerCoreInitFailuresTest, TestSearcherReuse, MultiThreadedOCPTest, TestFieldResource, TestLRUCache, TestPHPSerializedResponseWriter, TestStressReorder, ShardRoutingTest, TestFieldCollectionResource, BinaryUpdateRequestHandlerTest, LegacyHTMLStripCharFilterTest, TestChildDocTransformer, DirectSolrConnectionTest, SignatureUpdateProcessorFactoryTest, TestBlobHandler, SolrCoreCheckLockOnStartupTest, QueryResultKeyTest, RankQueryTest, BlockDirectoryTest, TestPseudoReturnFields, DistributedSuggestComponentTest, TestPhraseSuggestions, JsonLoaderTest, TestSolrXmlPersistor, TestStandardQParsers, ModifyConfFileTest, MigrateRouteKeyTest, SimpleMLTQParserTest, TestReload, AtomicUpdatesTest, CursorPagingTest, TestSolrIndexConfig, TestReplicationHandlerBackup, SpellCheckComponentTest, TestHighFrequencyDictionaryFactory, TestFreeTextSuggestions, HdfsCollectionsAPIDistributedZkTest, TestRandomMergePolicy, ReturnFieldsTest, TestSort, SharedFSAutoReplicaFailoverUtilsTest, DisMaxRequestHandlerTest, MBeansHandlerTest, TestSolrConfigHandler, TestInitQParser, TestCloudPivotFacet, CursorMarkTest, CopyFieldTest, TestJmxIntegration, ChaosMonkeyNothingIsSafeTest, HdfsSyncSliceTest, TestCollationFieldDocValues, SolrCmdDistributorTest, TestXIncludeConfig, TestAtomicUpdateErrorCases, UpdateParamsTest, TestLeaderElectionZkExpiry, TestPerFieldSimilarity, BasicDistributedZkTest, CurrencyFieldXmlFileTest, TestCollapseQParserPlugin, TestIndexSearcher, TestCloudInspectUtil, HdfsBasicDistributedZkTest, TestSearchPerf, WordBreakSolrSpellCheckerTest, CoreMergeIndexesAdminHandlerTest, CircularListTest, TestSolrDeletionPolicy1, TestCopyFieldCollectionResource, TestSolrQueryParserResource, OpenCloseCoreStressTest, DocValuesTest, SpellPossibilityIteratorTest, CloudExitableDirectoryReaderTest, TestManagedSchema, AssignTest, TestSchemaManager, DirectSolrSpellCheckerTest, TestRTGBase, TestStressVersions, PingRequestHandlerTest, SharedFSAutoReplicaFailoverTest, TestAddFieldRealTimeGet, SOLR749Test, UnloadDistributedZkTest, TestSolrConfigHandlerCloud, HttpPartitionTest, PeerSyncTest, AddBlockUpdateTest, TestRandomFaceting, TestRecoveryHdfs, TestNoOpRegenerator, PrimitiveFieldTypeTest, TestQuerySenderNoQuery, SuggestComponentTest, TestSolrConfigHandlerConcurrent, TestShardHandlerFactory, DistributedFacetPivotLargeTest, FastVectorHighlighterTest, TestPivotHelperCode, ClusterStateTest, SpatialFilterTest, StandardRequestHandlerTest, SimplePostToolTest, TestSolr4Spatial, DocumentBuilderTest, TestFastWriter, TestSerializedLuceneMatchVersion, HdfsLockFactoryTest, SearchHandlerTest, SolrIndexSplitterTest, ShardRoutingCustomTest, TestSystemIdResolver, TestReloadAndDeleteDocs, TestValueSourceCache, TestManagedStopFilterFactory, DeleteReplicaTest]
   [junit4] Completed in 56.76s, 1 test, 1 failure <<< FAILURES!

[...truncated 908 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:519: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:467: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:61: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:187: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:510: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1349: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:956: There were test failures: 452 suites, 1843 tests, 1 failure, 37 ignored (21 assumptions)

Total time: 164 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC (asserts: true)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Mime
View raw message