lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 1085 - Still Failing
Date Wed, 20 Mar 2013 14:42:36 GMT
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/1085/

1 tests failed.
FAILED:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:10964/collection1 Dead Guy:http://127.0.0.1:10958/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:10972/collection1], CloudJettyRunner [url=http://127.0.0.1:10972/collection1]]

Stack Trace:
java.lang.AssertionError: Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:10964/collection1 Dead Guy:http://127.0.0.1:10958/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:10972/collection1], CloudJettyRunner [url=http://127.0.0.1:10972/collection1]]
	at __randomizedtesting.SeedInfo.seed([F3334E0CFA380FF7:72D5C0148D676FCB]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:212)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9154 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4]   2> 0 T743 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 3 T743 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-SyncSliceTest-1363789533454
[junit4:junit4]   2> 4 T743 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 5 T744 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 8 T744 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 8 T744 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 8 T744 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 9 T744 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 9 T744 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1363789533454/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 106 T743 oasc.ZkTestServer.run start zk server on port:10947
[junit4:junit4]   2> 106 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1d52f58f
[junit4:junit4]   2> 107 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 108 T749 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 108 T749 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 108 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10948
[junit4:junit4]   2> 109 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10948
[junit4:junit4]   2> 109 T747 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 111 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d1140000 with negotiated timeout 10000 for client /140.211.11.196:10948
[junit4:junit4]   2> 111 T749 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d1140000, negotiated timeout = 10000
[junit4:junit4]   2> 112 T750 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d52f58f name:ZooKeeperConnection Watcher:127.0.0.1:10947 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 112 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 112 T743 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 115 T748 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8830d1140000
[junit4:junit4]   2> 116 T750 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 116 T743 oaz.ZooKeeper.close Session: 0x13d8830d1140000 closed
[junit4:junit4]   2> 116 T745 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10948 which had sessionid 0x13d8830d1140000
[junit4:junit4]   2> 117 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@14cf43b9
[junit4:junit4]   2> 117 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 118 T751 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 118 T751 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 118 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10949
[junit4:junit4]   2> 119 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10949
[junit4:junit4]   2> 125 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d1140001 with negotiated timeout 10000 for client /140.211.11.196:10949
[junit4:junit4]   2> 125 T751 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d1140001, negotiated timeout = 10000
[junit4:junit4]   2> 125 T752 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14cf43b9 name:ZooKeeperConnection Watcher:127.0.0.1:10947/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 126 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 126 T743 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 130 T743 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 134 T743 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 136 T743 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 140 T743 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 140 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 152 T743 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 153 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 256 T743 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 257 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 260 T743 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 261 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 271 T743 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 271 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 275 T743 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 275 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 278 T743 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 279 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 283 T743 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 284 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 287 T743 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 288 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 291 T748 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8830d1140001
[junit4:junit4]   2> 291 T752 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 291 T745 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10949 which had sessionid 0x13d8830d1140001
[junit4:junit4]   2> 291 T743 oaz.ZooKeeper.close Session: 0x13d8830d1140001 closed
[junit4:junit4]   2> 491 T743 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 498 T743 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10950
[junit4:junit4]   2> 498 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 499 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 499 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363789533742
[junit4:junit4]   2> 500 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363789533742/solr.xml
[junit4:junit4]   2> 500 T743 oasc.CoreContainer.<init> New CoreContainer 2004343128
[junit4:junit4]   2> 501 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363789533742/'
[junit4:junit4]   2> 502 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363789533742/'
[junit4:junit4]   2> 566 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 567 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 568 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 568 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 569 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 569 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 569 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 570 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 570 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 571 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 595 T743 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 615 T743 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:10947/solr
[junit4:junit4]   2> 616 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 617 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1fc25b02
[junit4:junit4]   2> 618 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 618 T762 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 619 T762 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 619 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10951
[junit4:junit4]   2> 619 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10951
[junit4:junit4]   2> 626 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d1140002 with negotiated timeout 20000 for client /140.211.11.196:10951
[junit4:junit4]   2> 626 T762 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d1140002, negotiated timeout = 20000
[junit4:junit4]   2> 627 T763 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fc25b02 name:ZooKeeperConnection Watcher:127.0.0.1:10947 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 627 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 628 T748 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8830d1140002
[junit4:junit4]   2> 635 T763 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 635 T745 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10951 which had sessionid 0x13d8830d1140002
[junit4:junit4]   2> 635 T743 oaz.ZooKeeper.close Session: 0x13d8830d1140002 closed
[junit4:junit4]   2> 636 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 650 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3b602327
[junit4:junit4]   2> 651 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 651 T764 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 652 T764 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 652 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10952
[junit4:junit4]   2> 652 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10952
[junit4:junit4]   2> 654 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d1140003 with negotiated timeout 20000 for client /140.211.11.196:10952
[junit4:junit4]   2> 654 T764 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d1140003, negotiated timeout = 20000
[junit4:junit4]   2> 654 T765 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b602327 name:ZooKeeperConnection Watcher:127.0.0.1:10947/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 654 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 657 T743 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 665 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140003 type:create cxid:0x7 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 666 T743 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 671 T743 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 677 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10950_
[junit4:junit4]   2> 678 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140003 type:delete cxid:0x12 zxid:0x1e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:10950_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:10950_
[junit4:junit4]   2> 679 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10950_
[junit4:junit4]   2> 683 T743 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 694 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140003 type:delete cxid:0x20 zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 696 T743 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 705 T743 oasc.Overseer.start Overseer (id=89377310864834563-127.0.0.1:10950_-n_0000000000) starting
[junit4:junit4]   2> 706 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140003 type:create cxid:0x25 zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 707 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140003 type:create cxid:0x26 zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 714 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140003 type:create cxid:0x28 zxid:0x27 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 715 T743 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 718 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140003 type:create cxid:0x2e zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 727 T767 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 728 T743 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 731 T743 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 733 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 736 T766 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 740 T768 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363789533742/collection1
[junit4:junit4]   2> 740 T768 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 741 T768 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 741 T768 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 742 T768 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363789533742/collection1/'
[junit4:junit4]   2> 744 T768 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1363789533742/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 744 T768 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1363789533742/collection1/lib/README' to classloader
[junit4:junit4]   2> 795 T768 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 870 T768 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 971 T768 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 978 T768 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1642 T768 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1653 T768 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1669 T768 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1686 T768 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1691 T768 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1695 T768 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1697 T768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1697 T768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 1697 T768 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1698 T768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1699 T768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 1699 T768 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1734 T768 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363789533742/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data/
[junit4:junit4]   2> 1734 T768 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@16fe1b92
[junit4:junit4]   2> 1735 T768 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1736 T768 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data forceNew: false
[junit4:junit4]   2> 1737 T768 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data/index/
[junit4:junit4]   2> 1737 T768 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1738 T768 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data/index forceNew: false
[junit4:junit4]   2> 1746 T768 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@11bbd441; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1747 T768 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1750 T768 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1750 T768 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1751 T768 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1751 T768 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1752 T768 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1752 T768 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1752 T768 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1753 T768 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1753 T768 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1766 T768 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1774 T768 oass.SolrIndexSearcher.<init> Opening Searcher@3cec558 main
[junit4:junit4]   2> 1775 T768 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1776 T768 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1779 T769 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3cec558 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1781 T768 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1781 T768 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2241 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2241 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10950_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10950"}
[junit4:junit4]   2> 2241 T766 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2242 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2254 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2784 T768 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2784 T768 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10950 collection:control_collection shard:shard1
[junit4:junit4]   2> 2785 T768 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2803 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140003 type:delete cxid:0x82 zxid:0x35 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2804 T768 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2805 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140003 type:create cxid:0x83 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2815 T768 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2816 T768 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2816 T768 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10950/collection1/
[junit4:junit4]   2> 2816 T768 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2817 T768 oasc.SyncStrategy.syncToMe http://127.0.0.1:10950/collection1/ has no replicas
[junit4:junit4]   2> 2817 T768 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10950/collection1/
[junit4:junit4]   2> 2817 T768 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2823 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140003 type:create cxid:0x8e zxid:0x3a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3760 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3777 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3803 T768 oasc.ZkController.register We are http://127.0.0.1:10950/collection1/ and leader is http://127.0.0.1:10950/collection1/
[junit4:junit4]   2> 3803 T768 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10950
[junit4:junit4]   2> 3803 T768 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3803 T768 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3804 T768 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3806 T768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3807 T743 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3807 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3808 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3819 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3820 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@43161c55
[junit4:junit4]   2> 3821 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3821 T771 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3822 T771 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 3822 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10957
[junit4:junit4]   2> 3822 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10957
[junit4:junit4]   2> 3824 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d1140004 with negotiated timeout 10000 for client /140.211.11.196:10957
[junit4:junit4]   2> 3824 T771 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d1140004, negotiated timeout = 10000
[junit4:junit4]   2> 3824 T772 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43161c55 name:ZooKeeperConnection Watcher:127.0.0.1:10947/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3824 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3826 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3827 T743 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3957 T743 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3960 T743 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10958
[junit4:junit4]   2> 3960 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3960 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3961 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363789537278
[junit4:junit4]   2> 3961 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363789537278/solr.xml
[junit4:junit4]   2> 3961 T743 oasc.CoreContainer.<init> New CoreContainer 774457874
[junit4:junit4]   2> 3962 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363789537278/'
[junit4:junit4]   2> 3962 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363789537278/'
[junit4:junit4]   2> 4005 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 4006 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4006 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4006 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4006 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4007 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4007 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4007 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4008 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4008 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4024 T743 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4038 T743 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:10947/solr
[junit4:junit4]   2> 4038 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4039 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4ca97517
[junit4:junit4]   2> 4039 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4040 T782 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4041 T782 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 4041 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10959
[junit4:junit4]   2> 4041 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10959
[junit4:junit4]   2> 4043 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d1140005 with negotiated timeout 20000 for client /140.211.11.196:10959
[junit4:junit4]   2> 4043 T782 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d1140005, negotiated timeout = 20000
[junit4:junit4]   2> 4043 T783 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ca97517 name:ZooKeeperConnection Watcher:127.0.0.1:10947 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4043 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4045 T748 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8830d1140005
[junit4:junit4]   2> 4050 T783 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4050 T745 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10959 which had sessionid 0x13d8830d1140005
[junit4:junit4]   2> 4050 T743 oaz.ZooKeeper.close Session: 0x13d8830d1140005 closed
[junit4:junit4]   2> 4051 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 4060 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@691e89a2
[junit4:junit4]   2> 4061 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4061 T784 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4062 T784 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 4062 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10960
[junit4:junit4]   2> 4062 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10960
[junit4:junit4]   2> 4064 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d1140006 with negotiated timeout 20000 for client /140.211.11.196:10960
[junit4:junit4]   2> 4064 T784 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d1140006, negotiated timeout = 20000
[junit4:junit4]   2> 4064 T785 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@691e89a2 name:ZooKeeperConnection Watcher:127.0.0.1:10947/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4064 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4066 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140006 type:create cxid:0x1 zxid:0x48 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4068 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140006 type:create cxid:0x3 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4076 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5079 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10958_
[junit4:junit4]   2> 5081 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140006 type:delete cxid:0xd zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:10958_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:10958_
[junit4:junit4]   2> 5082 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10958_
[junit4:junit4]   2> 5086 T785 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5086 T772 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5086 T765 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5086 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5103 T786 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363789537278/collection1
[junit4:junit4]   2> 5104 T786 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5104 T786 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5104 T786 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5106 T786 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363789537278/collection1/'
[junit4:junit4]   2> 5107 T786 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1363789537278/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5108 T786 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1363789537278/collection1/lib/README' to classloader
[junit4:junit4]   2> 5158 T786 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 5218 T786 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5285 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5285 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:10950__collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10950_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10950"}
[junit4:junit4]   2> 5298 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5298 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5319 T786 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5319 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5326 T786 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5993 T786 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6003 T786 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6007 T786 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6023 T786 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6028 T786 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6033 T786 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6035 T786 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 6035 T786 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 6035 T786 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6037 T786 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 6037 T786 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 6037 T786 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 6037 T786 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363789537278/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/
[junit4:junit4]   2> 6038 T786 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@16fe1b92
[junit4:junit4]   2> 6038 T786 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 6039 T786 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1 forceNew: false
[junit4:junit4]   2> 6040 T786 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index/
[junit4:junit4]   2> 6040 T786 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6041 T786 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index forceNew: false
[junit4:junit4]   2> 6047 T786 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6047 T786 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 6050 T786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6051 T786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6051 T786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6052 T786 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6053 T786 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6053 T786 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6053 T786 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6054 T786 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6054 T786 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6066 T786 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6074 T786 oass.SolrIndexSearcher.<init> Opening Searcher@c481f1a main
[junit4:junit4]   2> 6075 T786 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6075 T786 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6080 T787 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c481f1a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6082 T786 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 6082 T786 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6803 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6803 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10958_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10958"}
[junit4:junit4]   2> 6804 T766 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 6804 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6842 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6843 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6843 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7085 T786 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7085 T786 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10958 collection:collection1 shard:shard1
[junit4:junit4]   2> 7086 T786 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7142 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140006 type:delete cxid:0x4a zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7143 T786 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7143 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140006 type:create cxid:0x4b zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7157 T786 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7157 T786 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7157 T786 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10958/collection1/
[junit4:junit4]   2> 7158 T786 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7158 T786 oasc.SyncStrategy.syncToMe http://127.0.0.1:10958/collection1/ has no replicas
[junit4:junit4]   2> 7158 T786 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10958/collection1/
[junit4:junit4]   2> 7158 T786 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7163 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140006 type:create cxid:0x56 zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8347 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8364 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8364 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8364 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8398 T786 oasc.ZkController.register We are http://127.0.0.1:10958/collection1/ and leader is http://127.0.0.1:10958/collection1/
[junit4:junit4]   2> 8398 T786 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10958
[junit4:junit4]   2> 8398 T786 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8398 T786 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8398 T786 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8400 T786 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8401 T743 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 8402 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8402 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8546 T743 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8549 T743 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10964
[junit4:junit4]   2> 8549 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8550 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8550 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363789541864
[junit4:junit4]   2> 8550 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363789541864/solr.xml
[junit4:junit4]   2> 8551 T743 oasc.CoreContainer.<init> New CoreContainer 1783117957
[junit4:junit4]   2> 8551 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363789541864/'
[junit4:junit4]   2> 8552 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363789541864/'
[junit4:junit4]   2> 8594 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 8595 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8595 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 8596 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8596 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8596 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8597 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8597 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8597 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8598 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 8613 T743 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8627 T743 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:10947/solr
[junit4:junit4]   2> 8628 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 8629 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@ad719c9
[junit4:junit4]   2> 8630 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8630 T798 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8631 T798 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 8631 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10965
[junit4:junit4]   2> 8631 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10965
[junit4:junit4]   2> 8632 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d1140007 with negotiated timeout 20000 for client /140.211.11.196:10965
[junit4:junit4]   2> 8633 T798 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d1140007, negotiated timeout = 20000
[junit4:junit4]   2> 8633 T799 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ad719c9 name:ZooKeeperConnection Watcher:127.0.0.1:10947 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8633 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8634 T748 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8830d1140007
[junit4:junit4]   2> 8635 T799 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 8635 T745 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10965 which had sessionid 0x13d8830d1140007
[junit4:junit4]   2> 8635 T743 oaz.ZooKeeper.close Session: 0x13d8830d1140007 closed
[junit4:junit4]   2> 8636 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 8645 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@58741a85
[junit4:junit4]   2> 8646 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8646 T800 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8646 T800 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 8647 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10966
[junit4:junit4]   2> 8647 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10966
[junit4:junit4]   2> 8648 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d1140008 with negotiated timeout 20000 for client /140.211.11.196:10966
[junit4:junit4]   2> 8648 T800 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d1140008, negotiated timeout = 20000
[junit4:junit4]   2> 8648 T801 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58741a85 name:ZooKeeperConnection Watcher:127.0.0.1:10947/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8648 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8649 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8651 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140008 type:create cxid:0x3 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8661 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9664 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10964_
[junit4:junit4]   2> 9666 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d1140008 type:delete cxid:0xd zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:10964_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:10964_
[junit4:junit4]   2> 9667 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10964_
[junit4:junit4]   2> 9702 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9702 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9703 T765 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9703 T801 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9704 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9705 T785 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9705 T772 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9712 T802 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363789541864/collection1
[junit4:junit4]   2> 9712 T802 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9713 T802 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9713 T802 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9714 T802 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363789541864/collection1/'
[junit4:junit4]   2> 9715 T802 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1363789541864/collection1/lib/README' to classloader
[junit4:junit4]   2> 9716 T802 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1363789541864/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9768 T802 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 9843 T802 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9880 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9881 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:10958__collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10958_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10958"}
[junit4:junit4]   2> 9902 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9902 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9902 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9944 T802 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9944 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9951 T802 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10631 T802 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10643 T802 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10647 T802 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10664 T802 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10673 T802 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10680 T802 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10682 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 10683 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 10683 T802 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10685 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 10685 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 10686 T802 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 10686 T802 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363789541864/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/
[junit4:junit4]   2> 10686 T802 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@16fe1b92
[junit4:junit4]   2> 10687 T802 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 10689 T802 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2 forceNew: false
[junit4:junit4]   2> 10690 T802 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index/
[junit4:junit4]   2> 10691 T802 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10692 T802 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index forceNew: false
[junit4:junit4]   2> 10705 T802 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f54aef4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10705 T802 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 10710 T802 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10711 T802 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10712 T802 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10713 T802 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10714 T802 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10715 T802 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10715 T802 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10716 T802 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10717 T802 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10740 T802 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10754 T802 oass.SolrIndexSearcher.<init> Opening Searcher@76955a07 main
[junit4:junit4]   2> 10756 T802 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10756 T802 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10763 T803 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@76955a07 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10767 T802 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 10767 T802 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11408 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11409 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10964_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10964"}
[junit4:junit4]   2> 11409 T766 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 11410 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11422 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11422 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11422 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11422 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11770 T802 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 11770 T802 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10964 collection:collection1 shard:shard1
[junit4:junit4]   2> 11773 T802 oasc.ZkController.register We are http://127.0.0.1:10964/collection1/ and leader is http://127.0.0.1:10958/collection1/
[junit4:junit4]   2> 11773 T802 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10964
[junit4:junit4]   2> 11774 T802 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 11774 T802 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C63 name=collection1 org.apache.solr.core.SolrCore@5bfaff85 url=http://127.0.0.1:10964/collection1 node=127.0.0.1:10964_ C63_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:10964_, base_url=http://127.0.0.1:10964}
[junit4:junit4]   2> 11774 T804 C63 P10964 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 11775 T802 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11775 T804 C63 P10964 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 11776 T804 C63 P10964 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 11776 T804 C63 P10964 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11776 T743 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 11777 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11778 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11778 T804 C63 P10964 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12000 T743 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12003 T743 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10972
[junit4:junit4]   2> 12004 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12004 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12005 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363789545252
[junit4:junit4]   2> 12005 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363789545252/solr.xml
[junit4:junit4]   2> 12006 T743 oasc.CoreContainer.<init> New CoreContainer 1946731567
[junit4:junit4]   2> 12006 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363789545252/'
[junit4:junit4]   2> 12007 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363789545252/'
[junit4:junit4]   2> 12071 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 12072 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12072 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12073 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12073 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12074 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12074 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12075 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12075 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12076 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12099 T743 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12119 T743 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:10947/solr
[junit4:junit4]   2> 12120 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12120 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@300fbffa
[junit4:junit4]   2> 12121 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12122 T815 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12123 T815 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 12123 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10973
[junit4:junit4]   2> 12123 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10973
[junit4:junit4]   2> 12125 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d1140009 with negotiated timeout 20000 for client /140.211.11.196:10973
[junit4:junit4]   2> 12125 T815 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d1140009, negotiated timeout = 20000
[junit4:junit4]   2> 12126 T816 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@300fbffa name:ZooKeeperConnection Watcher:127.0.0.1:10947 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12126 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12127 T748 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8830d1140009
[junit4:junit4]   2> 12129 T816 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12129 T745 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10973 which had sessionid 0x13d8830d1140009
[junit4:junit4]   2> 12129 T743 oaz.ZooKeeper.close Session: 0x13d8830d1140009 closed
[junit4:junit4]   2> 12129 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 12143 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@30481ae8
[junit4:junit4]   2> 12144 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12144 T817 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12145 T817 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 12145 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10974
[junit4:junit4]   2> 12145 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10974
[junit4:junit4]   2> 12147 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d114000a with negotiated timeout 20000 for client /140.211.11.196:10974
[junit4:junit4]   2> 12147 T817 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d114000a, negotiated timeout = 20000
[junit4:junit4]   2> 12147 T818 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30481ae8 name:ZooKeeperConnection Watcher:127.0.0.1:10947/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12147 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12148 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d114000a type:create cxid:0x1 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12150 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d114000a type:create cxid:0x3 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12153 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12927 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12928 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:10964__collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10964_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10964"}
[junit4:junit4]   2> 12941 T818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12941 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12941 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12941 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12941 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13156 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10972_
[junit4:junit4]   2> 13158 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d114000a type:delete cxid:0xf zxid:0x85 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:10972_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:10972_
[junit4:junit4]   2> 13159 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10972_
[junit4:junit4]   2> 13161 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13162 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13162 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13162 T818 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13162 T765 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13162 T818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13162 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13164 T801 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13164 T785 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13164 T772 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13176 T819 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363789545252/collection1
[junit4:junit4]   2> 13176 T819 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13177 T819 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13177 T819 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13178 T819 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363789545252/collection1/'
[junit4:junit4]   2> 13180 T819 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1363789545252/collection1/lib/README' to classloader
[junit4:junit4]   2> 13180 T819 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1363789545252/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13233 T819 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 13293 T819 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13394 T819 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13401 T819 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 13807 T775 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:10964__collection1&state=recovering&nodeName=127.0.0.1:10964_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002 
[junit4:junit4]   2> 14101 T819 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14113 T819 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14118 T819 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14134 T819 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14140 T819 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14145 T819 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14146 T819 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 14147 T819 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 14147 T819 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14148 T819 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 14149 T819 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 14149 T819 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 14149 T819 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363789545252/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/
[junit4:junit4]   2> 14150 T819 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@16fe1b92
[junit4:junit4]   2> 14150 T819 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 14151 T819 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3 forceNew: false
[junit4:junit4]   2> 14152 T819 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index/
[junit4:junit4]   2> 14153 T819 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14154 T819 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index forceNew: false
[junit4:junit4]   2> 14160 T819 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@31674127; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14161 T819 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 14164 T819 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14164 T819 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14165 T819 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14166 T819 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14166 T819 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14167 T819 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14167 T819 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14168 T819 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14168 T819 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14183 T819 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14192 T819 oass.SolrIndexSearcher.<init> Opening Searcher@511af6db main
[junit4:junit4]   2> 14193 T819 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14194 T819 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14199 T820 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@511af6db main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14201 T819 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 14201 T819 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14448 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14449 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10972_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10972"}
[junit4:junit4]   2> 14449 T766 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 14449 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 14477 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14477 T818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14477 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14477 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14477 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15204 T819 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15204 T819 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10972 collection:collection1 shard:shard1
[junit4:junit4]   2> 15208 T819 oasc.ZkController.register We are http://127.0.0.1:10972/collection1/ and leader is http://127.0.0.1:10958/collection1/
[junit4:junit4]   2> 15209 T819 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10972
[junit4:junit4]   2> 15209 T819 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 15209 T819 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C64 name=collection1 org.apache.solr.core.SolrCore@21b7b8f7 url=http://127.0.0.1:10972/collection1 node=127.0.0.1:10972_ C64_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:10972_, base_url=http://127.0.0.1:10972}
[junit4:junit4]   2> 15209 T821 C64 P10972 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 15210 T821 C64 P10972 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 15210 T819 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15211 T821 C64 P10972 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 15211 T821 C64 P10972 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15212 T743 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 15212 T821 C64 P10972 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15212 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15213 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15426 T743 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15429 T743 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10986
[junit4:junit4]   2> 15430 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15430 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15431 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363789548684
[junit4:junit4]   2> 15431 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363789548684/solr.xml
[junit4:junit4]   2> 15432 T743 oasc.CoreContainer.<init> New CoreContainer 856607126
[junit4:junit4]   2> 15432 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363789548684/'
[junit4:junit4]   2> 15433 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363789548684/'
[junit4:junit4]   2> 15497 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 15498 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15498 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15499 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15499 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15499 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15500 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15500 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15501 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15501 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15523 T743 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15543 T743 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:10947/solr
[junit4:junit4]   2> 15544 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15544 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@564feb91
[junit4:junit4]   2> 15545 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15546 T832 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15546 T832 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 15547 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10987
[junit4:junit4]   2> 15547 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10987
[junit4:junit4]   2> 15549 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d114000b with negotiated timeout 20000 for client /140.211.11.196:10987
[junit4:junit4]   2> 15549 T832 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d114000b, negotiated timeout = 20000
[junit4:junit4]   2> 15549 T833 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@564feb91 name:ZooKeeperConnection Watcher:127.0.0.1:10947 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15549 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15551 T748 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d8830d114000b
[junit4:junit4]   2> 15552 T833 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15552 T745 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10987 which had sessionid 0x13d8830d114000b
[junit4:junit4]   2> 15552 T743 oaz.ZooKeeper.close Session: 0x13d8830d114000b closed
[junit4:junit4]   2> 15552 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 15564 T743 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:10947/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@cd02eb7
[junit4:junit4]   2> 15565 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15565 T834 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:10947. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15566 T834 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:10947, initiating session
[junit4:junit4]   2> 15566 T745 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10988
[junit4:junit4]   2> 15566 T745 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10988
[junit4:junit4]   2> 15567 T747 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d8830d114000c with negotiated timeout 20000 for client /140.211.11.196:10988
[junit4:junit4]   2> 15567 T834 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:10947, sessionid = 0x13d8830d114000c, negotiated timeout = 20000
[junit4:junit4]   2> 15568 T835 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cd02eb7 name:ZooKeeperConnection Watcher:127.0.0.1:10947/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15568 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15569 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d114000c type:create cxid:0x1 zxid:0x92 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15574 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d114000c type:create cxid:0x3 zxid:0x93 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15576 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2>  C63_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:10964_, base_url=http://127.0.0.1:10964}
[junit4:junit4]   2> 15809 T804 C63 P10964 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:10958/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15809 T804 C63 P10964 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:10964 START replicas=[http://127.0.0.1:10958/collection1/] nUpdates=100
[junit4:junit4]   2> 15810 T804 C63 P10964 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15810 T804 C63 P10964 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15811 T804 C63 P10964 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15811 T804 C63 P10964 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15811 T804 C63 P10964 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15811 T804 C63 P10964 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:10958/collection1/. core=collection1
[junit4:junit4]   2> 15811 T804 C63 P10964 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C65 name=collection1 org.apache.solr.core.SolrCore@11be6ed0 url=http://127.0.0.1:10958/collection1 node=127.0.0.1:10958_ C65_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:10958_, base_url=http://127.0.0.1:10958, leader=true}
[junit4:junit4]   2> 15819 T777 C65 P10958 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15825 T778 C65 P10958 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 15830 T778 C65 P10958 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15831 T778 C65 P10958 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15838 T778 C65 P10958 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15838 T778 C65 P10958 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15840 T778 C65 P10958 oass.SolrIndexSearcher.<init> Opening Searcher@244ec142 realtime
[junit4:junit4]   2> 15841 T778 C65 P10958 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15841 T778 C65 P10958 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 16
[junit4:junit4]   2> 15842 T804 C63 P10964 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15842 T804 C63 P10964 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15845 T779 C65 P10958 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15846 T779 C65 P10958 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15847 T804 C63 P10964 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15847 T804 C63 P10964 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15847 T804 C63 P10964 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15850 T779 C65 P10958 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15851 T804 C63 P10964 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15852 T804 C63 P10964 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index.20130320182549302 forceNew: false
[junit4:junit4]   2> 15854 T804 C63 P10964 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index.20130320182549302 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18723646; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4]   2> 15857 T779 C65 P10958 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 15859 T804 C63 P10964 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15863 T804 C63 P10964 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15863 T804 C63 P10964 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 15868 T804 C63 P10964 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f54aef4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f54aef4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15869 T804 C63 P10964 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15869 T804 C63 P10964 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 15870 T804 C63 P10964 oass.SolrIndexSearcher.<init> Opening Searcher@1737c526 main
[junit4:junit4]   2> 15871 T803 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1737c526 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 15872 T804 C63 P10964 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 15872 T804 C63 P10964 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 15872 T804 C63 P10964 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15872 T804 C63 P10964 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15874 T804 C63 P10964 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 15983 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15985 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:10972__collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10972_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10972"}
[junit4:junit4]   2> 15990 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:10964__collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10964_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10964"}
[junit4:junit4]   2> 16012 T818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16013 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16012 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16013 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16013 T835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16013 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16237 T776 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:10972__collection1&state=recovering&nodeName=127.0.0.1:10972_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001 
[junit4:junit4]   2> 16580 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10986_
[junit4:junit4]   2> 16581 T748 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d8830d114000c type:delete cxid:0xf zxid:0x9c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:10986_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:10986_
[junit4:junit4]   2> 16585 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10986_
[junit4:junit4]   2> 16588 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16588 T835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16588 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16588 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16602 T765 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16589 T818 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16602 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16603 T818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16604 T772 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16604 T785 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16604 T835 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16605 T801 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16611 T838 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363789548684/collection1
[junit4:junit4]   2> 16611 T838 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 16612 T838 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 16612 T838 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 16613 T838 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363789548684/collection1/'
[junit4:junit4]   2> 16615 T838 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1363789548684/collection1/lib/README' to classloader
[junit4:junit4]   2> 16615 T838 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1363789548684/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 16692 T838 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 16752 T838 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 16853 T838 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 16860 T838 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 17537 T838 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 17549 T838 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 17553 T838 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 17569 T838 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17574 T838 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17579 T838 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17580 T838 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 17581 T838 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 17581 T838 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17582 T838 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 17582 T838 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 17583 T838 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 17583 T838 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363789548684/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/
[junit4:junit4]   2> 17583 T838 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@16fe1b92
[junit4:junit4]   2> 17584 T838 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 17585 T838 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4 forceNew: false
[junit4:junit4]   2> 17585 T838 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index/
[junit4:junit4]   2> 17586 T838 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 17587 T838 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index forceNew: false
[junit4:junit4]   2> 17594 T838 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@556aa027; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 17595 T838 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 17598 T838 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 17598 T838 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 17599 T838 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 17600 T838 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 17600 T838 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 17600 T838 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 17601 T838 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 17601 T838 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 17602 T838 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 17616 T838 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 17624 T838 oass.SolrIndexSearcher.<init> Opening Searcher@27323315 main
[junit4:junit4]   2> 17626 T838 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 17626 T838 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 17630 T839 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@27323315 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 17633 T838 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 17633 T838 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2>  C64_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:10972_, base_url=http://127.0.0.1:10972}
[junit4:junit4]   2> 18239 T821 C64 P10972 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:10958/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18239 T821 C64 P10972 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:10972 START replicas=[http://127.0.0.1:10958/collection1/] nUpdates=100
[junit4:junit4]   2> 18239 T821 C64 P10972 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18240 T821 C64 P10972 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18240 T821 C64 P10972 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18240 T821 C64 P10972 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18240 T821 C64 P10972 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18240 T821 C64 P10972 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:10958/collection1/. core=collection1
[junit4:junit4]   2> 18241 T821 C64 P10972 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18242 T777 C65 P10958 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18253 T780 C65 P10958 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 18257 T780 C65 P10958 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18257 T780 C65 P10958 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18258 T780 C65 P10958 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18258 T780 C65 P10958 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4]   2> 18259 T821 C64 P10972 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18260 T821 C64 P10972 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18262 T779 C65 P10958 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18263 T821 C64 P10972 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4]   2> 18263 T821 C64 P10972 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18263 T821 C64 P10972 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18266 T779 C65 P10958 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18266 T821 C64 P10972 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18267 T821 C64 P10972 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index.20130320182551717 forceNew: false
[junit4:junit4]   2> 18268 T821 C64 P10972 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index.20130320182551717 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3a8c12cf; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4]   2> 18272 T779 C65 P10958 REQ /replication {file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
[junit4:junit4]   2> 18273 T821 C64 P10972 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18275 T821 C64 P10972 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18276 T821 C64 P10972 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18278 T821 C64 P10972 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@31674127; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@31674127; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18279 T821 C64 P10972 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18279 T821 C64 P10972 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18279 T821 C64 P10972 oass.SolrIndexSearcher.<init> Opening Searcher@1163adab main
[junit4:junit4]   2> 18281 T820 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1163adab main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4]   2> 18281 T821 C64 P10972 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18281 T821 C64 P10972 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18281 T821 C64 P10972 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18281 T821 C64 P10972 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18283 T821 C64 P10972 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 19024 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19025 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10986_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10986"}
[junit4:junit4]   2> 19025 T766 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 19025 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 19030 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:10972__collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10972_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10972"}
[junit4:junit4]   2> 19042 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19042 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19042 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19042 T818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19042 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19042 T835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19637 T838 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 19637 T838 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10986 collection:collection1 shard:shard1
[junit4:junit4]   2> 19664 T838 oasc.ZkController.register We are http://127.0.0.1:10986/collection1/ and leader is http://127.0.0.1:10958/collection1/
[junit4:junit4]   2> 19664 T838 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10986
[junit4:junit4]   2> 19664 T838 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 19664 T838 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C66 name=collection1 org.apache.solr.core.SolrCore@2e4bc07a url=http://127.0.0.1:10986/collection1 node=127.0.0.1:10986_ C66_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:10986_, base_url=http://127.0.0.1:10986}
[junit4:junit4]   2> 19665 T842 C66 P10986 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 19665 T838 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19666 T842 C66 P10986 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 19667 T842 C66 P10986 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 19667 T842 C66 P10986 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19667 T743 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 19668 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 19669 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19683 T842 C66 P10986 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19688 T743 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19689 T743 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 19690 T743 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 19690 T743 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20557 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20558 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:10986__collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10986_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10986"}
[junit4:junit4]   2> 20569 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20569 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20569 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20569 T835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20569 T818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20569 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20692 T743 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20699 T781 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:10986__collection1&state=recovering&nodeName=127.0.0.1:10986_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001 
[junit4:junit4]   2> 21694 T743 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22697 T743 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C66_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:10986_, base_url=http://127.0.0.1:10986}
[junit4:junit4]   2> 22701 T842 C66 P10986 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:10958/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 22701 T842 C66 P10986 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:10986 START replicas=[http://127.0.0.1:10958/collection1/] nUpdates=100
[junit4:junit4]   2> 22701 T842 C66 P10986 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 22702 T842 C66 P10986 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 22702 T842 C66 P10986 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 22702 T842 C66 P10986 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 22702 T842 C66 P10986 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 22702 T842 C66 P10986 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:10958/collection1/. core=collection1
[junit4:junit4]   2> 22703 T842 C66 P10986 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 22705 T777 C65 P10958 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 22718 T775 C65 P10958 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 22722 T775 C65 P10958 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 22723 T775 C65 P10958 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 22724 T775 C65 P10958 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 22724 T775 C65 P10958 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 22726 T842 C66 P10986 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 22726 T842 C66 P10986 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 22728 T779 C65 P10958 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 22729 T842 C66 P10986 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4]   2> 22729 T842 C66 P10986 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 22730 T842 C66 P10986 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 22733 T779 C65 P10958 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1 
[junit4:junit4]   2> 22734 T842 C66 P10986 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 22735 T842 C66 P10986 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index.20130320182556184 forceNew: false
[junit4:junit4]   2> 22736 T842 C66 P10986 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index.20130320182556184 lockFactory=org.apache.lucene.store.NativeFSLockFactory@684e384f; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4]   2> 22739 T779 C65 P10958 REQ /replication {file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
[junit4:junit4]   2> 22741 T842 C66 P10986 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 22743 T842 C66 P10986 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 22743 T842 C66 P10986 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 22747 T842 C66 P10986 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@556aa027; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@556aa027; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 22748 T842 C66 P10986 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 22748 T842 C66 P10986 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 22749 T842 C66 P10986 oass.SolrIndexSearcher.<init> Opening Searcher@5caa2283 main
[junit4:junit4]   2> 22750 T839 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5caa2283 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4]   2> 22751 T842 C66 P10986 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 22751 T842 C66 P10986 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 22751 T842 C66 P10986 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 22752 T842 C66 P10986 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22753 T842 C66 P10986 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 23581 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23582 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:10986__collection1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:10986_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:10986"}
[junit4:junit4]   2> 23588 T818 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23588 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23588 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23588 T835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23588 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23588 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23700 T743 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23702 T743 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C67 name=collection1 org.apache.solr.core.SolrCore@b159ebd url=http://127.0.0.1:10950/collection1 node=127.0.0.1:10950_ C67_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:10950_, base_url=http://127.0.0.1:10950, leader=true}
[junit4:junit4]   2> 23713 T755 C67 P10950 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 23718 T755 C67 P10950 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@11bbd441; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 23718 T755 C67 P10950 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 23723 T755 C67 P10950 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@11bbd441; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@11bbd441; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 23723 T755 C67 P10950 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 23727 T755 C67 P10950 oass.SolrIndexSearcher.<init> Opening Searcher@374dc2b4 main
[junit4:junit4]   2> 23727 T755 C67 P10950 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23728 T769 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@374dc2b4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 23729 T755 C67 P10950 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 16
[junit4:junit4]   2> 23742 T778 C65 P10958 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 23746 T778 C65 P10958 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=4
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 23747 T778 C65 P10958 oasc.SolrDeletionPolicy.updateCommits newest commit = 5[segments_5]
[junit4:junit4]   2> 23748 T778 C65 P10958 oass.SolrIndexSearcher.<init> Opening Searcher@38e15d63 main
[junit4:junit4]   2> 23749 T778 C65 P10958 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23750 T787 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@38e15d63 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 23750 T778 C65 P10958 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10964/collection1/, StdNode: http://127.0.0.1:10972/collection1/, StdNode: http://127.0.0.1:10986/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2>  C64_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:10972_, base_url=http://127.0.0.1:10972}
[junit4:junit4]   2> 23767 T808 C64 P10972 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 23773 T808 C64 P10972 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@31674127; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@31674127; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 23773 T808 C64 P10972 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 23776 T808 C64 P10972 oass.SolrIndexSearcher.<init> Opening Searcher@2dadfcbe main
[junit4:junit4]   2> 23776 T808 C64 P10972 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2>  C66_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:10986_, base_url=http://127.0.0.1:10986}
[junit4:junit4]   2> 23777 T825 C66 P10986 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2>  C63_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:10964_, base_url=http://127.0.0.1:10964}
[junit4:junit4]   2> 23777 T791 C63 P10964 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 23777 T820 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2dadfcbe main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4]   2> 23778 T808 C64 P10972 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4]   2> 23783 T791 C63 P10964 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f54aef4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f54aef4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 23783 T825 C66 P10986 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@556aa027; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@556aa027; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 23784 T791 C63 P10964 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 23784 T825 C66 P10986 oasc.SolrDeletionPolicy.updateCommits newest commit = 5[segments_5]
[junit4:junit4]   2> 23787 T791 C63 P10964 oass.SolrIndexSearcher.<init> Opening Searcher@2ba56d9 main
[junit4:junit4]   2> 23787 T825 C66 P10986 oass.SolrIndexSearcher.<init> Opening Searcher@179bc1f5 main
[junit4:junit4]   2> 23787 T791 C63 P10964 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23787 T825 C66 P10986 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23788 T803 oasc.JmxMonitoredMap.put WARNING Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4]   2> 	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4]   2> 	at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4]   2> 	at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4]   2> 	at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:239)
[junit4:junit4]   2> 	at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1763)
[junit4:junit4]   2> 	at org.apache.solr.core.SolrCore.access$000(SolrCore.java:129)
[junit4:junit4]   2> 	at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1640)
[junit4:junit4]   2> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 23788 T839 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@179bc1f5 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4]   2> 23789 T825 C66 P10986 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4]   2> 23789 T803 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2ba56d9 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 23790 T791 C63 P10964 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4]   2> 23791 T778 C65 P10958 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 49
[junit4:junit4]   2> 23792 T743 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23795 T776 C65 P10958 REQ /select {tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 23799 T792 C63 P10964 REQ /select {tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 23801 T809 C64 P10972 REQ /select {tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 23804 T826 C66 P10986 REQ /select {tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 25810 T755 C67 P10950 oass.SolrIndexSearcher.<init> Opening Searcher@542ab85a realtime
[junit4:junit4]   2> 25810 T755 C67 P10950 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1430037000888516608)} 0 3
[junit4:junit4]   2> 25820 T825 C66 P10986 /update {update.distrib=FROMLEADER&_version_=-1430037000894808064&update.from=http://127.0.0.1:10958/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1430037000894808064)} 0 2
[junit4:junit4]   2> 25820 T791 C63 P10964 /update {update.distrib=FROMLEADER&_version_=-1430037000894808064&update.from=http://127.0.0.1:10958/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1430037000894808064)} 0 1
[junit4:junit4]   2> 25820 T808 C64 P10972 /update {update.distrib=FROMLEADER&_version_=-1430037000894808064&update.from=http://127.0.0.1:10958/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1430037000894808064)} 0 2
[junit4:junit4]   2> 25821 T778 C65 P10958 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1430037000894808064)} 0 8
[junit4:junit4]   2> 25827 T755 C67 P10950 /update {wt=javabin&version=2} {add=[0 (1430037000905293824)]} 0 3
[junit4:junit4]   2> 25838 T808 C64 P10972 /update {distrib.from=http://127.0.0.1:10958/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1430037000911585280)]} 0 3
[junit4:junit4]   2> 25838 T825 C66 P10986 /update {distrib.from=http://127.0.0.1:10958/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1430037000911585280)]} 0 3
[junit4:junit4]   2> 25838 T791 C63 P10964 /update {distrib.from=http://127.0.0.1:10958/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1430037000911585280)]} 0 3
[junit4:junit4]   2> 25839 T778 C65 P10958 /update {wt=javabin&version=2} {add=[0 (1430037000911585280)]} 0 9
[junit4:junit4]   2> 25843 T755 C67 P10950 /update {wt=javabin&version=2} {add=[1 (1430037000924168192)]} 0 1
[junit4:junit4]   2> 25851 T825 C66 P10986 /update {distrib.from=http://127.0.0.1:10958/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1430037000928362496)]} 0 1
[junit4:junit4]   2> 25851 T791 C63 P10964 /update {distrib.from=http://127.0.0.1:10958/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1430037000928362496)]} 0 1
[junit4:junit4]   2> 25852 T808 C64 P10972 /update {distrib.from=http://127.0.0.1:10958/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1430037000928362496)]} 0 2
[junit4:junit4]   2> 25852 T778 C65 P10958 /update {wt=javabin&version=2} {add=[1 (1430037000928362496)]} 0 6
[junit4:junit4]   2> 25856 T755 C67 P10950 /update {wt=javabin&version=2} {add=[2 (1430037000937799680)]} 0 1
[junit4:junit4]   2> 25864 T808 C64 P10972 /update {distrib.from=http://127.0.0.1:10958/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1430037000943042560)]} 0 1
[junit4:junit4]   2> 25864 T825 C66 P10986 /update {distrib.from=http://127.0.0.1:10958/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1430037000943042560)]} 0 1
[junit4:junit4]   2> 25865 T778 C65 P10958 /update {test.distrib.skip.servers=http://127.0.0.1:10964/collection1/&wt=javabin&version=2} {add=[2 (1430037000943042560)]} 0 6
[junit4:junit4]   2> 25869 T755 C67 P10950 /update {wt=javabin&version=2} {add=[3 (1430037000951431168)]} 0 1
[junit4:junit4]   2> 25877 T825 C66 P10986 /update {distrib.from=http://127.0.0.1:10958/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1430037000955625472)]} 0 1
[junit4:junit4]   2> 25878 T778 C65 P10958 /update {test.distrib.skip.servers=http://127.0.0.1:10964/collection1/&test.distrib.skip.servers=http://127.0.0.1:10972/collection1/&wt=javabin&version=2} {add=[3 (1430037000955625472)]} 0 6
[junit4:junit4]   2> 25881 T755 C67 P10950 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 25934 T755 C67 P10950 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@11bbd441; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@11bbd441; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3]
[junit4:junit4]   2> 25935 T755 C67 P10950 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3]
[junit4:junit4]   2> 25941 T755 C67 P10950 oass.SolrIndexSearcher.<init> Opening Searcher@31da0621 main
[junit4:junit4]   2> 25942 T755 C67 P10950 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 25943 T769 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31da0621 main{StandardDirectoryReader(segments_3:3:nrt _0(4.3):C4)}
[junit4:junit4]   2> 25943 T755 C67 P10950 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 62
[junit4:junit4]   2> 25946 T778 C65 P10958 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 25994 T778 C65 P10958 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=5
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36acd8f7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, segments_6, _0.fdt, _0_Lucene41_0.tip]
[junit4:junit4]   2> 25995 T778 C65 P10958 oasc.SolrDeletionPolicy.updateCommits newest commit = 6[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, segments_6, _0.fdt, _0_Lucene41_0.tip]
[junit4:junit4]   2> 26001 T778 C65 P10958 oass.SolrIndexSearcher.<init> Opening Searcher@33e707c4 main
[junit4:junit4]   2> 26002 T778 C65 P10958 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 26003 T787 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@33e707c4 main{StandardDirectoryReader(segments_6:3:nrt _0(4.3):C4)}
[junit4:junit4]   2> 26003 T778 C65 P10958 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10964/collection1/, StdNode: http://127.0.0.1:10972/collection1/, StdNode: http://127.0.0.1:10986/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 26006 T808 C64 P10972 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 26006 T825 C66 P10986 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 26007 T791 C63 P10964 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 26115 T808 C64 P10972 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@31674127; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@31674127; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, segments_5, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip]
[junit4:junit4]   2> 26115 T825 C66 P10986 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@556aa027; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@556aa027; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, segments_6, _0.fdt, _0_Lucene41_0.tip]
[junit4:junit4]   2> 26115 T825 C66 P10986 oasc.SolrDeletionPolicy.updateCommits newest commit = 6[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, segments_6, _0.fdt, _0_Lucene41_0.tip]
[junit4:junit4]   2> 26115 T808 C64 P10972 oasc.SolrDeletionPolicy.updateCommits newest commit = 5[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_nrm.cfs, _0.si, segments_5, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip]
[junit4:junit4]   2> 26116 T791 C63 P10964 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f54aef4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1363789533454/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f54aef4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segF

[...truncated too long message...]

g... (live nodes size: 2)
[junit4:junit4]   2> 142740 T818 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 142740 T745 oazs.NIOServerCnxnFactory.run NIOServerCnxn factory exited run method
[junit4:junit4]   2> 142740 T818 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 142741 T743 oazs.ZooKeeperServer.shutdown shutting down
[junit4:junit4]   2> 142742 T743 oazs.SessionTrackerImpl.shutdown Shutting down
[junit4:junit4]   2> 142742 T743 oazs.PrepRequestProcessor.shutdown Shutting down
[junit4:junit4]   2> 142742 T743 oazs.SyncRequestProcessor.shutdown Shutting down
[junit4:junit4]   2> 142743 T743 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=F3334E0CFA380FF7 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ko_KR -Dtests.timezone=Asia/Tbilisi -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE  143s J1 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:10964/collection1 Dead Guy:http://127.0.0.1:10958/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:10972/collection1], CloudJettyRunner [url=http://127.0.0.1:10972/collection1]]
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([F3334E0CFA380FF7:72D5C0148D676FCB]:0)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.junit.Assert.assertTrue(Assert.java:43)
[junit4:junit4]    > 	at org.junit.Assert.assertNotNull(Assert.java:526)
[junit4:junit4]    > 	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:212)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 142764 T743 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=ko_KR, timezone=Asia/Tbilisi
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_17 (64-bit)/cpus=16,threads=5,free=119895856,total=192937984
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestHashPartitioner, MBeansHandlerTest, CurrencyFieldOpenExchangeTest, ShowFileRequestHandlerTest, PeerSyncTest, SimpleFacetsTest, TestPHPSerializedResponseWriter, TestJmxMonitoredMap, TestSolrDeletionPolicy2, TestZkChroot, TestFastWriter, TestDynamicFieldResource, PathHierarchyTokenizerFactoryTest, TestDocSet, TestBinaryField, DirectSolrConnectionTest, RecoveryZkTest, OutputWriterTest, TestRecovery, ZkNodePropsTest, CacheHeaderTest, TestQuerySenderNoQuery, StatelessScriptUpdateProcessorFactoryTest, TestValueSourceCache, CoreAdminHandlerTest, FieldMutatingUpdateProcessorTest, TestFieldResource, TestJoin, TestFastLRUCache, UpdateParamsTest, SpellCheckComponentTest, QueryElevationComponentTest, DocumentBuilderTest, SpellCheckCollatorTest, TestFieldTypeCollectionResource, TestPartialUpdateDeduplication, TestLuceneMatchVersion, TestFunctionQuery, TestBinaryResponseWriter, TestCollationKeyRangeQueries, DateFieldTest, DOMUtilTest, TestRemoteStreaming, TestSweetSpotSimilarityFactory, ResponseLogComponentTest, TestOmitPositions, TestExtendedDismaxParser, IndexReaderFactoryTest, DocValuesTest, LegacyHTMLStripCharFilterTest, ExternalFileFieldSortTest, TestRangeQuery, SystemInfoHandlerTest, ShardRoutingCustomTest, TestPseudoReturnFields, TestSolrDiscoveryProperties, ZkCLITest, StatsComponentTest, SynonymTokenizerTest, SolrInfoMBeanTest, TestCSVResponseWriter, HighlighterConfigTest, StandardRequestHandlerTest, SyncSliceTest]
[junit4:junit4] Completed on J1 in 142.79s, 1 test, 1 failure <<< FAILURES!

[...truncated 561 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:877: There were test failures: 273 suites, 1147 tests, 1 failure, 27 ignored (8 assumptions)

Total time: 47 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message