lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 208 - Still Failing
Date Thu, 14 Mar 2013 20:49:44 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/208/

1 tests failed.
FAILED:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
shard1 is not consistent.  Got 305 from http://127.0.0.1:13698/collection1lastClient and got 5 from http://127.0.0.1:13704/collection1

Stack Trace:
java.lang.AssertionError: shard1 is not consistent.  Got 305 from http://127.0.0.1:13698/collection1lastClient and got 5 from http://127.0.0.1:13704/collection1
	at __randomizedtesting.SeedInfo.seed([7ADAB6D4D7D0DD92:FB3C38CCA08FBDAE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:963)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:235)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:679)




Build Log:
[...truncated 8813 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4]   2> 1 T2671 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 7 T2671 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1363293371509
[junit4:junit4]   2> 8 T2671 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 9 T2672 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 14 T2672 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 14 T2672 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 15 T2672 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 15 T2672 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 17 T2672 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1363293371508/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 110 T2671 oasc.ZkTestServer.run start zk server on port:13653
[junit4:junit4]   2> 111 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@eee2993
[junit4:junit4]   2> 112 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 112 T2677 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 113 T2677 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 113 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13654
[junit4:junit4]   2> 113 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13654
[junit4:junit4]   2> 114 T2675 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 118 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0000 with negotiated timeout 10000 for client /140.211.11.196:13654
[junit4:junit4]   2> 118 T2677 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0000, negotiated timeout = 10000
[junit4:junit4]   2> 119 T2678 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@eee2993 name:ZooKeeperConnection Watcher:127.0.0.1:13653 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 119 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 119 T2671 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 134 T2676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6a9dfc7f0000
[junit4:junit4]   2> 136 T2678 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 137 T2673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13654 which had sessionid 0x13d6a9dfc7f0000
[junit4:junit4]   2> 136 T2671 oaz.ZooKeeper.close Session: 0x13d6a9dfc7f0000 closed
[junit4:junit4]   2> 138 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@425998ab
[junit4:junit4]   2> 138 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 139 T2679 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 139 T2679 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 140 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13655
[junit4:junit4]   2> 140 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13655
[junit4:junit4]   2> 142 T2679 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0001, negotiated timeout = 10000
[junit4:junit4]   2> 142 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0001 with negotiated timeout 10000 for client /140.211.11.196:13655
[junit4:junit4]   2> 143 T2680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@425998ab name:ZooKeeperConnection Watcher:127.0.0.1:13653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 143 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 144 T2671 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 150 T2671 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 154 T2671 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 159 T2671 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 165 T2671 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 167 T2671 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 182 T2671 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 183 T2671 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 288 T2671 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 289 T2671 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 299 T2671 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 300 T2671 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 304 T2671 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 305 T2671 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 309 T2671 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 310 T2671 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 314 T2671 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 314 T2671 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 318 T2671 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 319 T2671 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 322 T2671 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 323 T2671 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 326 T2676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6a9dfc7f0001
[junit4:junit4]   2> 327 T2680 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 327 T2673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13655 which had sessionid 0x13d6a9dfc7f0001
[junit4:junit4]   2> 327 T2671 oaz.ZooKeeper.close Session: 0x13d6a9dfc7f0001 closed
[junit4:junit4]   2> 561 T2671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 566 T2671 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13656
[junit4:junit4]   2> 567 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 567 T2671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 568 T2671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363293371829
[junit4:junit4]   2> 568 T2671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363293371829/solr.xml
[junit4:junit4]   2> 569 T2671 oasc.CoreContainer.<init> New CoreContainer 1007816338
[junit4:junit4]   2> 569 T2671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363293371829/'
[junit4:junit4]   2> 570 T2671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363293371829/'
[junit4:junit4]   2> 642 T2671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 643 T2671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 643 T2671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 644 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 644 T2671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 645 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 645 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 646 T2671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 646 T2671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 647 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 671 T2671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 693 T2671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13653/solr
[junit4:junit4]   2> 694 T2671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 694 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7a3f971d
[junit4:junit4]   2> 696 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 696 T2690 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 697 T2690 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 697 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13657
[junit4:junit4]   2> 698 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13657
[junit4:junit4]   2> 703 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0002 with negotiated timeout 20000 for client /140.211.11.196:13657
[junit4:junit4]   2> 703 T2690 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0002, negotiated timeout = 20000
[junit4:junit4]   2> 703 T2691 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a3f971d name:ZooKeeperConnection Watcher:127.0.0.1:13653 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 703 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 705 T2676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6a9dfc7f0002
[junit4:junit4]   2> 714 T2691 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 714 T2673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13657 which had sessionid 0x13d6a9dfc7f0002
[junit4:junit4]   2> 714 T2671 oaz.ZooKeeper.close Session: 0x13d6a9dfc7f0002 closed
[junit4:junit4]   2> 714 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 727 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@2cf4dafb
[junit4:junit4]   2> 728 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 729 T2692 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 729 T2692 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 730 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13658
[junit4:junit4]   2> 730 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13658
[junit4:junit4]   2> 732 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0003 with negotiated timeout 20000 for client /140.211.11.196:13658
[junit4:junit4]   2> 732 T2692 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0003, negotiated timeout = 20000
[junit4:junit4]   2> 733 T2693 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cf4dafb name:ZooKeeperConnection Watcher:127.0.0.1:13653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 733 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 735 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 737 T2671 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 742 T2671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13656_
[junit4:junit4]   2> 742 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:13656_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:13656_
[junit4:junit4]   2> 744 T2671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:13656_
[junit4:junit4]   2> 754 T2671 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 766 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 772 T2671 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 775 T2671 oasc.Overseer.start Overseer (id=89344794395869187-127.0.0.1:13656_-n_0000000000) starting
[junit4:junit4]   2> 776 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 778 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 786 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 788 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 789 T2695 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 790 T2671 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 804 T2671 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 807 T2671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 811 T2694 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 814 T2696 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363293371829/collection1
[junit4:junit4]   2> 815 T2696 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 815 T2696 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 815 T2696 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 817 T2696 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363293371829/collection1/'
[junit4:junit4]   2> 818 T2696 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1363293371829/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 819 T2696 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1363293371829/collection1/lib/README' to classloader
[junit4:junit4]   2> 878 T2696 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 952 T2696 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1056 T2696 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1064 T2696 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1819 T2696 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1827 T2696 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1831 T2696 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1845 T2696 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1849 T2696 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1853 T2696 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1854 T2696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1854 T2696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 1854 T2696 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1856 T2696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1856 T2696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 1856 T2696 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1856 T2696 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363293371829/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/control/data/
[junit4:junit4]   2> 1856 T2696 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2658dd2d
[junit4:junit4]   2> 1857 T2696 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1857 T2696 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/control/data forceNew: false
[junit4:junit4]   2> 1857 T2696 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363293371508/control/data/index/
[junit4:junit4]   2> 1857 T2696 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363293371508/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1858 T2696 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/control/data/index forceNew: false
[junit4:junit4]   2> 1860 T2696 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3055a3d6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@42fd5260),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1860 T2696 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1862 T2696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1862 T2696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1862 T2696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1863 T2696 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1863 T2696 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1863 T2696 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1863 T2696 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1864 T2696 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1864 T2696 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1869 T2696 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1873 T2696 oass.SolrIndexSearcher.<init> Opening Searcher@118cedce main
[junit4:junit4]   2> 1873 T2696 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/control/data/tlog
[junit4:junit4]   2> 1874 T2696 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1874 T2696 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1877 T2697 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@118cedce main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1878 T2696 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1878 T2696 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1879 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:create cxid:0x55 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2316 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2316 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13656_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13656"}
[junit4:junit4]   2> 2317 T2694 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2317 T2694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2317 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:create cxid:0x5d zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2336 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2894 T2696 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2894 T2696 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:13656 collection:control_collection shard:shard1
[junit4:junit4]   2> 2895 T2696 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2915 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:delete cxid:0x77 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2916 T2696 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2916 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:create cxid:0x78 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2920 T2696 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2920 T2696 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2920 T2696 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:13656/collection1/
[junit4:junit4]   2> 2920 T2696 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2921 T2696 oasc.SyncStrategy.syncToMe http://127.0.0.1:13656/collection1/ has no replicas
[junit4:junit4]   2> 2921 T2696 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:13656/collection1/
[junit4:junit4]   2> 2921 T2696 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2927 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0003 type:create cxid:0x82 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3842 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3890 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3949 T2696 oasc.ZkController.register We are http://127.0.0.1:13656/collection1/ and leader is http://127.0.0.1:13656/collection1/
[junit4:junit4]   2> 3949 T2696 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:13656
[junit4:junit4]   2> 3949 T2696 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3950 T2696 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3950 T2696 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3952 T2696 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3954 T2671 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3955 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3955 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3971 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3973 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@720116b6
[junit4:junit4]   2> 3973 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3974 T2699 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 3975 T2699 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 3975 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13660
[junit4:junit4]   2> 3975 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13660
[junit4:junit4]   2> 3977 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0004 with negotiated timeout 10000 for client /140.211.11.196:13660
[junit4:junit4]   2> 3977 T2699 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0004, negotiated timeout = 10000
[junit4:junit4]   2> 3977 T2700 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@720116b6 name:ZooKeeperConnection Watcher:127.0.0.1:13653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3977 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3979 T2671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3981 T2671 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4193 T2671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4196 T2671 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13661
[junit4:junit4]   2> 4197 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4198 T2671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4198 T2671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363293375483
[junit4:junit4]   2> 4198 T2671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363293375483/solr.xml
[junit4:junit4]   2> 4199 T2671 oasc.CoreContainer.<init> New CoreContainer 292605757
[junit4:junit4]   2> 4200 T2671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363293375483/'
[junit4:junit4]   2> 4200 T2671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363293375483/'
[junit4:junit4]   2> 4260 T2671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 4261 T2671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4262 T2671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4262 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4263 T2671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4263 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4264 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4264 T2671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4265 T2671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4265 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4287 T2671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4308 T2671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13653/solr
[junit4:junit4]   2> 4309 T2671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4310 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@97034f7
[junit4:junit4]   2> 4311 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4311 T2710 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 4312 T2710 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 4313 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13662
[junit4:junit4]   2> 4313 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13662
[junit4:junit4]   2> 4315 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0005 with negotiated timeout 20000 for client /140.211.11.196:13662
[junit4:junit4]   2> 4315 T2710 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0005, negotiated timeout = 20000
[junit4:junit4]   2> 4315 T2711 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@97034f7 name:ZooKeeperConnection Watcher:127.0.0.1:13653 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4316 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4317 T2676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6a9dfc7f0005
[junit4:junit4]   2> 4326 T2711 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4326 T2673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13662 which had sessionid 0x13d6a9dfc7f0005
[junit4:junit4]   2> 4326 T2671 oaz.ZooKeeper.close Session: 0x13d6a9dfc7f0005 closed
[junit4:junit4]   2> 4327 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 4336 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@58f587bb
[junit4:junit4]   2> 4337 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4338 T2712 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 4338 T2712 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 4338 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13663
[junit4:junit4]   2> 4339 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13663
[junit4:junit4]   2> 4340 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0006 with negotiated timeout 20000 for client /140.211.11.196:13663
[junit4:junit4]   2> 4340 T2712 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0006, negotiated timeout = 20000
[junit4:junit4]   2> 4341 T2713 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58f587bb name:ZooKeeperConnection Watcher:127.0.0.1:13653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4341 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4342 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0006 type:create cxid:0x1 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4344 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0006 type:create cxid:0x2 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4347 T2671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5353 T2671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13661_
[junit4:junit4]   2> 5359 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0006 type:delete cxid:0xb zxid:0x4c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:13661_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:13661_
[junit4:junit4]   2> 5361 T2671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:13661_
[junit4:junit4]   2> 5381 T2693 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5381 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5386 T2713 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5388 T2700 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5427 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5432 T2714 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363293375483/collection1
[junit4:junit4]   2> 5432 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13656_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13656"}
[junit4:junit4]   2> 5432 T2714 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5434 T2714 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5435 T2714 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5450 T2714 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363293375483/collection1/'
[junit4:junit4]   2> 5459 T2714 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1363293375483/collection1/lib/README' to classloader
[junit4:junit4]   2> 5463 T2714 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1363293375483/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5470 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5470 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5471 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5700 T2714 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 5811 T2714 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5912 T2714 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5920 T2714 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6655 T2714 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6666 T2714 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6671 T2714 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6687 T2714 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6693 T2714 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6698 T2714 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6700 T2714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 6701 T2714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 6701 T2714 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6702 T2714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 6703 T2714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 6703 T2714 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 6703 T2714 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363293375483/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty1/
[junit4:junit4]   2> 6704 T2714 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2658dd2d
[junit4:junit4]   2> 6704 T2714 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 6705 T2714 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty1 forceNew: false
[junit4:junit4]   2> 6705 T2714 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty1/index/
[junit4:junit4]   2> 6705 T2714 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6706 T2714 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty1/index forceNew: false
[junit4:junit4]   2> 6708 T2714 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61f6247e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b339fce),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6709 T2714 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 6711 T2714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6711 T2714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6712 T2714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6712 T2714 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6713 T2714 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6713 T2714 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6714 T2714 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6714 T2714 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6715 T2714 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6722 T2714 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6728 T2714 oass.SolrIndexSearcher.<init> Opening Searcher@452c8361 main
[junit4:junit4]   2> 6728 T2714 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty1/tlog
[junit4:junit4]   2> 6729 T2714 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6729 T2714 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6733 T2715 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@452c8361 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6735 T2714 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 6735 T2714 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6987 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6989 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13661_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13661"}
[junit4:junit4]   2> 6989 T2694 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 6989 T2694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6996 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6996 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6996 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7739 T2714 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7739 T2714 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:13661 collection:collection1 shard:shard1
[junit4:junit4]   2> 7741 T2714 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7762 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0006 type:delete cxid:0x48 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7764 T2714 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7765 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0006 type:create cxid:0x49 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7776 T2714 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7776 T2714 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7776 T2714 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:13661/collection1/
[junit4:junit4]   2> 7777 T2714 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7777 T2714 oasc.SyncStrategy.syncToMe http://127.0.0.1:13661/collection1/ has no replicas
[junit4:junit4]   2> 7777 T2714 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:13661/collection1/
[junit4:junit4]   2> 7777 T2714 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7783 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0006 type:create cxid:0x53 zxid:0x61 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8508 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8552 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8552 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8552 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8561 T2714 oasc.ZkController.register We are http://127.0.0.1:13661/collection1/ and leader is http://127.0.0.1:13661/collection1/
[junit4:junit4]   2> 8561 T2714 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:13661
[junit4:junit4]   2> 8561 T2714 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8561 T2714 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8562 T2714 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8564 T2714 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8565 T2671 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8566 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8566 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8801 T2671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8803 T2671 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13665
[junit4:junit4]   2> 8805 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8805 T2671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8806 T2671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363293380084
[junit4:junit4]   2> 8806 T2671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363293380084/solr.xml
[junit4:junit4]   2> 8807 T2671 oasc.CoreContainer.<init> New CoreContainer 537842846
[junit4:junit4]   2> 8807 T2671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363293380084/'
[junit4:junit4]   2> 8808 T2671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363293380084/'
[junit4:junit4]   2> 8864 T2671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 8865 T2671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8865 T2671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 8866 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8866 T2671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8867 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8867 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8868 T2671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8868 T2671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8869 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 8893 T2671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8914 T2671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13653/solr
[junit4:junit4]   2> 8915 T2671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 8916 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2a6d76e0
[junit4:junit4]   2> 8917 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8917 T2726 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 8918 T2726 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 8919 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13666
[junit4:junit4]   2> 8919 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13666
[junit4:junit4]   2> 8921 T2726 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0007, negotiated timeout = 20000
[junit4:junit4]   2> 8921 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0007 with negotiated timeout 20000 for client /140.211.11.196:13666
[junit4:junit4]   2> 8922 T2727 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2a6d76e0 name:ZooKeeperConnection Watcher:127.0.0.1:13653 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8922 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8923 T2676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6a9dfc7f0007
[junit4:junit4]   2> 8925 T2673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13666 which had sessionid 0x13d6a9dfc7f0007
[junit4:junit4]   2> 8925 T2727 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 8925 T2671 oaz.ZooKeeper.close Session: 0x13d6a9dfc7f0007 closed
[junit4:junit4]   2> 8925 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 8938 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@66eb63f8
[junit4:junit4]   2> 8939 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8940 T2728 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 8940 T2728 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 8941 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13667
[junit4:junit4]   2> 8941 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13667
[junit4:junit4]   2> 8943 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0008 with negotiated timeout 20000 for client /140.211.11.196:13667
[junit4:junit4]   2> 8943 T2728 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0008, negotiated timeout = 20000
[junit4:junit4]   2> 8943 T2729 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66eb63f8 name:ZooKeeperConnection Watcher:127.0.0.1:13653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8943 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8944 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0008 type:create cxid:0x1 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8946 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0008 type:create cxid:0x2 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8955 T2671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9959 T2671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13665_
[junit4:junit4]   2> 9960 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0008 type:delete cxid:0xb zxid:0x70 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:13665_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:13665_
[junit4:junit4]   2> 9965 T2671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:13665_
[junit4:junit4]   2> 9979 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9979 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9979 T2729 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9979 T2693 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9979 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9980 T2700 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9980 T2713 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9987 T2730 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363293380084/collection1
[junit4:junit4]   2> 9988 T2730 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9988 T2730 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9988 T2730 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9990 T2730 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363293380084/collection1/'
[junit4:junit4]   2> 9991 T2730 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1363293380084/collection1/lib/README' to classloader
[junit4:junit4]   2> 9991 T2730 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1363293380084/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10048 T2730 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 10060 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10061 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13661_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13661"}
[junit4:junit4]   2> 10078 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10078 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10078 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10078 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10113 T2730 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10214 T2730 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10220 T2730 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10828 T2730 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10836 T2730 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10839 T2730 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10850 T2730 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10854 T2730 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10857 T2730 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10858 T2730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 10859 T2730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 10859 T2730 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10859 T2730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 10860 T2730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 10860 T2730 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 10860 T2730 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363293380084/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty2/
[junit4:junit4]   2> 10860 T2730 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2658dd2d
[junit4:junit4]   2> 10861 T2730 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 10861 T2730 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty2 forceNew: false
[junit4:junit4]   2> 10861 T2730 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty2/index/
[junit4:junit4]   2> 10861 T2730 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10862 T2730 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty2/index forceNew: false
[junit4:junit4]   2> 10863 T2730 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@653a0728 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a00792d),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10864 T2730 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 10865 T2730 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10865 T2730 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10866 T2730 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10866 T2730 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10867 T2730 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10867 T2730 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10867 T2730 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10867 T2730 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10868 T2730 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10874 T2730 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10878 T2730 oass.SolrIndexSearcher.<init> Opening Searcher@bfebbac main
[junit4:junit4]   2> 10878 T2730 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty2/tlog
[junit4:junit4]   2> 10879 T2730 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10879 T2730 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10882 T2731 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@bfebbac main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10884 T2730 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 10884 T2730 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11582 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11583 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13665_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13665"}
[junit4:junit4]   2> 11583 T2694 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 11583 T2694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11602 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11602 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11602 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11602 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11895 T2730 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 11895 T2730 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:13665 collection:collection1 shard:shard1
[junit4:junit4]   2> 11899 T2730 oasc.ZkController.register We are http://127.0.0.1:13665/collection1/ and leader is http://127.0.0.1:13661/collection1/
[junit4:junit4]   2> 11899 T2730 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:13665
[junit4:junit4]   2> 11900 T2730 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 11900 T2730 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C189 name=collection1 org.apache.solr.core.SolrCore@62219a0d url=http://127.0.0.1:13665/collection1 node=127.0.0.1:13665_ C189_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:13665_, base_url=http://127.0.0.1:13665}
[junit4:junit4]   2> 11900 T2732 C189 P13665 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 11901 T2730 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11901 T2732 C189 P13665 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 11901 T2732 C189 P13665 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 11901 T2732 C189 P13665 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11902 T2671 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11902 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11903 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11910 T2732 C189 P13665 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12119 T2671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12122 T2671 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13670
[junit4:junit4]   2> 12123 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12123 T2671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12124 T2671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363293383420
[junit4:junit4]   2> 12124 T2671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363293383420/solr.xml
[junit4:junit4]   2> 12124 T2671 oasc.CoreContainer.<init> New CoreContainer 813728562
[junit4:junit4]   2> 12125 T2671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363293383420/'
[junit4:junit4]   2> 12125 T2671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363293383420/'
[junit4:junit4]   2> 12176 T2671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 12177 T2671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12177 T2671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12178 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12178 T2671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12178 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12179 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12179 T2671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12180 T2671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12180 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12201 T2671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12220 T2671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13653/solr
[junit4:junit4]   2> 12220 T2671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12221 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@18e213a9
[junit4:junit4]   2> 12222 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12222 T2743 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 12223 T2743 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 12223 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13671
[junit4:junit4]   2> 12224 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13671
[junit4:junit4]   2> 12226 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0009 with negotiated timeout 20000 for client /140.211.11.196:13671
[junit4:junit4]   2> 12226 T2743 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0009, negotiated timeout = 20000
[junit4:junit4]   2> 12227 T2744 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18e213a9 name:ZooKeeperConnection Watcher:127.0.0.1:13653 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12227 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12228 T2676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6a9dfc7f0009
[junit4:junit4]   2> 12236 T2744 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12236 T2673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13671 which had sessionid 0x13d6a9dfc7f0009
[junit4:junit4]   2> 12236 T2671 oaz.ZooKeeper.close Session: 0x13d6a9dfc7f0009 closed
[junit4:junit4]   2> 12237 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 12248 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@51957d2
[junit4:junit4]   2> 12249 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12249 T2745 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 12250 T2745 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 12250 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13672
[junit4:junit4]   2> 12250 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13672
[junit4:junit4]   2> 12251 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f000a with negotiated timeout 20000 for client /140.211.11.196:13672
[junit4:junit4]   2> 12252 T2745 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f000a, negotiated timeout = 20000
[junit4:junit4]   2> 12252 T2746 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51957d2 name:ZooKeeperConnection Watcher:127.0.0.1:13653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12252 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12253 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f000a type:create cxid:0x1 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12254 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f000a type:create cxid:0x2 zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12257 T2671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13106 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13107 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13665_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13665"}
[junit4:junit4]   2> 13123 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13123 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13123 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13123 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13123 T2746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13260 T2671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13670_
[junit4:junit4]   2> 13261 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f000a type:delete cxid:0xd zxid:0x87 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:13670_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:13670_
[junit4:junit4]   2> 13262 T2671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:13670_
[junit4:junit4]   2> 13268 T2746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13268 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13268 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13268 T2729 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13268 T2693 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13268 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13269 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13269 T2746 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13269 T2700 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13271 T2713 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13276 T2747 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363293383420/collection1
[junit4:junit4]   2> 13276 T2747 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13277 T2747 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13277 T2747 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13278 T2747 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363293383420/collection1/'
[junit4:junit4]   2> 13279 T2747 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1363293383420/collection1/lib/README' to classloader
[junit4:junit4]   2> 13280 T2747 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1363293383420/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13325 T2747 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 13379 T2747 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13480 T2747 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13486 T2747 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 13925 T2703 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=3&state=recovering&nodeName=127.0.0.1:13665_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002 
[junit4:junit4]   2> 14026 T2747 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14035 T2747 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14039 T2747 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14052 T2747 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14059 T2747 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14065 T2747 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14067 T2747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 14067 T2747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 14068 T2747 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14069 T2747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 14069 T2747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 14069 T2747 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 14070 T2747 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363293383420/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty3/
[junit4:junit4]   2> 14070 T2747 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2658dd2d
[junit4:junit4]   2> 14071 T2747 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 14072 T2747 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty3 forceNew: false
[junit4:junit4]   2> 14072 T2747 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty3/index/
[junit4:junit4]   2> 14072 T2747 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14073 T2747 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty3/index forceNew: false
[junit4:junit4]   2> 14077 T2747 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12c6123d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e0f2b80),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14077 T2747 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 14080 T2747 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14081 T2747 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14081 T2747 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14082 T2747 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14083 T2747 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14083 T2747 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14083 T2747 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14084 T2747 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14085 T2747 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14102 T2747 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14117 T2747 oass.SolrIndexSearcher.<init> Opening Searcher@1ebccff8 main
[junit4:junit4]   2> 14117 T2747 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty3/tlog
[junit4:junit4]   2> 14119 T2747 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14119 T2747 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14124 T2748 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ebccff8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14127 T2747 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 14127 T2747 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14629 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14630 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13670_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13670"}
[junit4:junit4]   2> 14630 T2694 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 14630 T2694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 14648 T2746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14648 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14648 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14648 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14648 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15159 T2747 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15159 T2747 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:13670 collection:collection1 shard:shard1
[junit4:junit4]   2> 15163 T2747 oasc.ZkController.register We are http://127.0.0.1:13670/collection1/ and leader is http://127.0.0.1:13661/collection1/
[junit4:junit4]   2> 15163 T2747 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:13670
[junit4:junit4]   2> 15164 T2747 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 15164 T2747 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C190 name=collection1 org.apache.solr.core.SolrCore@516e17a5 url=http://127.0.0.1:13670/collection1 node=127.0.0.1:13670_ C190_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:13670_, base_url=http://127.0.0.1:13670}
[junit4:junit4]   2> 15164 T2749 C190 P13670 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 15165 T2747 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15165 T2749 C190 P13670 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 15165 T2749 C190 P13670 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 15166 T2749 C190 P13670 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15166 T2671 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 15167 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15168 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15173 T2749 C190 P13670 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15495 T2671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15499 T2671 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13691
[junit4:junit4]   2> 15500 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15500 T2671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15501 T2671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363293386703
[junit4:junit4]   2> 15501 T2671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363293386703/solr.xml
[junit4:junit4]   2> 15501 T2671 oasc.CoreContainer.<init> New CoreContainer 1782731087
[junit4:junit4]   2> 15502 T2671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363293386703/'
[junit4:junit4]   2> 15503 T2671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363293386703/'
[junit4:junit4]   2> 15577 T2671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 15578 T2671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15579 T2671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15579 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15579 T2671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15580 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15580 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15581 T2671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15581 T2671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15581 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15608 T2671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15631 T2671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13653/solr
[junit4:junit4]   2> 15631 T2671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15632 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4de9357
[junit4:junit4]   2> 15634 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15634 T2760 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 15635 T2760 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 15636 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13692
[junit4:junit4]   2> 15636 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13692
[junit4:junit4]   2> 15638 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f000b with negotiated timeout 20000 for client /140.211.11.196:13692
[junit4:junit4]   2> 15638 T2760 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f000b, negotiated timeout = 20000
[junit4:junit4]   2> 15639 T2761 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4de9357 name:ZooKeeperConnection Watcher:127.0.0.1:13653 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15639 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15640 T2676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6a9dfc7f000b
[junit4:junit4]   2> 15641 T2761 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15641 T2673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13692 which had sessionid 0x13d6a9dfc7f000b
[junit4:junit4]   2> 15641 T2671 oaz.ZooKeeper.close Session: 0x13d6a9dfc7f000b closed
[junit4:junit4]   2> 15642 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 15653 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3c31541b
[junit4:junit4]   2> 15654 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15654 T2762 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 15654 T2762 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 15655 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13693
[junit4:junit4]   2> 15655 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13693
[junit4:junit4]   2> 15656 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f000c with negotiated timeout 20000 for client /140.211.11.196:13693
[junit4:junit4]   2> 15656 T2762 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f000c, negotiated timeout = 20000
[junit4:junit4]   2> 15657 T2763 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c31541b name:ZooKeeperConnection Watcher:127.0.0.1:13653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15657 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15658 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f000c type:create cxid:0x1 zxid:0x94 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15659 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f000c type:create cxid:0x2 zxid:0x95 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15668 T2671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2>  C189_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:13665_, base_url=http://127.0.0.1:13665}
[junit4:junit4]   2> 15926 T2732 C189 P13665 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:13661/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15926 T2732 C189 P13665 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:13665 START replicas=[http://127.0.0.1:13661/collection1/] nUpdates=100
[junit4:junit4]   2> 15926 T2732 C189 P13665 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15927 T2732 C189 P13665 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15927 T2732 C189 P13665 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15927 T2732 C189 P13665 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15928 T2732 C189 P13665 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15928 T2732 C189 P13665 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:13661/collection1/. core=collection1
[junit4:junit4]   2> 15928 T2732 C189 P13665 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C191 name=collection1 org.apache.solr.core.SolrCore@278ece77 url=http://127.0.0.1:13661/collection1 node=127.0.0.1:13661_ C191_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:13661_, base_url=http://127.0.0.1:13661, leader=true}
[junit4:junit4]   2> 15935 T2705 C191 P13661 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15941 T2706 C191 P13661 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 15944 T2706 C191 P13661 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61f6247e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b339fce),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15945 T2706 C191 P13661 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15946 T2706 C191 P13661 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61f6247e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b339fce),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61f6247e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b339fce),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15946 T2706 C191 P13661 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15947 T2706 C191 P13661 oass.SolrIndexSearcher.<init> Opening Searcher@6b4fe04a realtime
[junit4:junit4]   2> 15947 T2706 C191 P13661 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15948 T2706 C191 P13661 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4]   2> 15950 T2732 C189 P13665 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15950 T2732 C189 P13665 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15952 T2707 C191 P13661 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15953 T2707 C191 P13661 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15953 T2732 C189 P13665 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15953 T2732 C189 P13665 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15953 T2732 C189 P13665 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15956 T2707 C191 P13661 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15957 T2732 C189 P13665 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15958 T2732 C189 P13665 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty2/index.20130314143627458 forceNew: false
[junit4:junit4]   2> 15958 T2732 C189 P13665 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@4ca1760a lockFactory=org.apache.lucene.store.NativeFSLockFactory@f2ed366) fullCopy=false
[junit4:junit4]   2> 15960 T2707 C191 P13661 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 15961 T2732 C189 P13665 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15971 T2732 C189 P13665 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15971 T2732 C189 P13665 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 16024 T2732 C189 P13665 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@653a0728 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a00792d),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@653a0728 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a00792d),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 16025 T2732 C189 P13665 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 16025 T2732 C189 P13665 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 16025 T2732 C189 P13665 oass.SolrIndexSearcher.<init> Opening Searcher@3dbd423e main
[junit4:junit4]   2> 16026 T2731 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3dbd423e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 16027 T2732 C189 P13665 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@4ca1760a lockFactory=org.apache.lucene.store.NativeFSLockFactory@f2ed366)
[junit4:junit4]   2> 16027 T2732 C189 P13665 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 16027 T2732 C189 P13665 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 16027 T2732 C189 P13665 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 16028 T2732 C189 P13665 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16062 T2732 C189 P13665 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 16159 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16159 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13670_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13670"}
[junit4:junit4]   2> 16201 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13665_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13665"}
[junit4:junit4]   2> 16289 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16289 T2746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16289 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16289 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16289 T2763 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16289 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16672 T2671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13691_
[junit4:junit4]   2> 16673 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f000c type:delete cxid:0xd zxid:0x9e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:13691_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:13691_
[junit4:junit4]   2> 16675 T2671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:13691_
[junit4:junit4]   2> 16677 T2746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16677 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16677 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16678 T2763 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16678 T2693 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16678 T2763 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16678 T2729 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16678 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16678 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16679 T2713 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16680 T2746 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16685 T2700 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16691 T2766 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363293386703/collection1
[junit4:junit4]   2> 16691 T2766 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 16692 T2766 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 16692 T2766 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 16693 T2766 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363293386703/collection1/'
[junit4:junit4]   2> 16695 T2766 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1363293386703/collection1/lib/README' to classloader
[junit4:junit4]   2> 16696 T2766 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1363293386703/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 16750 T2766 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 16816 T2766 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 16917 T2766 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 16925 T2766 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 17206 T2704 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=4&state=recovering&nodeName=127.0.0.1:13670_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002 
[junit4:junit4]   2> 17660 T2766 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 17677 T2766 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 17682 T2766 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 17696 T2766 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17701 T2766 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17707 T2766 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17708 T2766 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 17708 T2766 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 17709 T2766 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17710 T2766 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 17710 T2766 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 17710 T2766 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 17711 T2766 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363293386703/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty4/
[junit4:junit4]   2> 17711 T2766 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2658dd2d
[junit4:junit4]   2> 17711 T2766 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 17712 T2766 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty4 forceNew: false
[junit4:junit4]   2> 17712 T2766 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty4/index/
[junit4:junit4]   2> 17712 T2766 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 17713 T2766 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty4/index forceNew: false
[junit4:junit4]   2> 17716 T2766 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@21917d40 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4841c79a),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 17717 T2766 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 17719 T2766 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 17719 T2766 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 17720 T2766 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 17721 T2766 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 17721 T2766 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 17721 T2766 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 17722 T2766 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 17722 T2766 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 17722 T2766 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 17733 T2766 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 17741 T2766 oass.SolrIndexSearcher.<init> Opening Searcher@417d0498 main
[junit4:junit4]   2> 17741 T2766 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty4/tlog
[junit4:junit4]   2> 17742 T2766 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 17742 T2766 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 17746 T2767 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@417d0498 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 17748 T2766 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 17749 T2766 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17801 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17801 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13691_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13691"}
[junit4:junit4]   2> 17802 T2694 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 17802 T2694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 17826 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17827 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17827 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17827 T2763 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17827 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17827 T2746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18752 T2766 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 18752 T2766 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:13691 collection:collection1 shard:shard1
[junit4:junit4]   2> 18756 T2766 oasc.ZkController.register We are http://127.0.0.1:13691/collection1/ and leader is http://127.0.0.1:13661/collection1/
[junit4:junit4]   2> 18757 T2766 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:13691
[junit4:junit4]   2> 18757 T2766 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 18757 T2766 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C192 name=collection1 org.apache.solr.core.SolrCore@6087febc url=http://127.0.0.1:13691/collection1 node=127.0.0.1:13691_ C192_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:13691_, base_url=http://127.0.0.1:13691}
[junit4:junit4]   2> 18757 T2768 C192 P13691 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 18758 T2768 C192 P13691 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 18758 T2766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18758 T2768 C192 P13691 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 18759 T2768 C192 P13691 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18759 T2671 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 18760 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 18760 T2768 C192 P13691 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18761 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19041 T2671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 19044 T2671 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13698
[junit4:junit4]   2> 19045 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 19045 T2671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 19046 T2671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363293390284
[junit4:junit4]   2> 19046 T2671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363293390284/solr.xml
[junit4:junit4]   2> 19047 T2671 oasc.CoreContainer.<init> New CoreContainer 1972326028
[junit4:junit4]   2> 19047 T2671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363293390284/'
[junit4:junit4]   2> 19048 T2671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363293390284/'
[junit4:junit4]   2> 19106 T2671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 19106 T2671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 19107 T2671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 19107 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 19108 T2671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 19108 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 19109 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 19109 T2671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 19109 T2671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 19110 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 19130 T2671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 19150 T2671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13653/solr
[junit4:junit4]   2> 19151 T2671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 19151 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c335557
[junit4:junit4]   2> 19152 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 19153 T2779 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 19154 T2779 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 19154 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13699
[junit4:junit4]   2> 19154 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13699
[junit4:junit4]   2> 19155 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f000d with negotiated timeout 20000 for client /140.211.11.196:13699
[junit4:junit4]   2> 19155 T2779 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f000d, negotiated timeout = 20000
[junit4:junit4]   2> 19156 T2780 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c335557 name:ZooKeeperConnection Watcher:127.0.0.1:13653 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 19156 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 19157 T2676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6a9dfc7f000d
[junit4:junit4]   2> 19158 T2780 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 19158 T2673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13699 which had sessionid 0x13d6a9dfc7f000d
[junit4:junit4]   2> 19158 T2671 oaz.ZooKeeper.close Session: 0x13d6a9dfc7f000d closed
[junit4:junit4]   2> 19158 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 19169 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1cd75ff9
[junit4:junit4]   2> 19170 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 19170 T2781 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 19171 T2781 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 19171 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13700
[junit4:junit4]   2> 19172 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13700
[junit4:junit4]   2> 19174 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f000e with negotiated timeout 20000 for client /140.211.11.196:13700
[junit4:junit4]   2> 19174 T2781 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f000e, negotiated timeout = 20000
[junit4:junit4]   2> 19174 T2782 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1cd75ff9 name:ZooKeeperConnection Watcher:127.0.0.1:13653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 19174 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 19175 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f000e type:create cxid:0x1 zxid:0xab txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 19182 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f000e type:create cxid:0x2 zxid:0xac txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 19189 T2671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2>  C190_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:13670_, base_url=http://127.0.0.1:13670}
[junit4:junit4]   2> 19208 T2749 C190 P13670 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:13661/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 19208 T2749 C190 P13670 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:13670 START replicas=[http://127.0.0.1:13661/collection1/] nUpdates=100
[junit4:junit4]   2> 19208 T2749 C190 P13670 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 19209 T2749 C190 P13670 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 19209 T2749 C190 P13670 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 19209 T2749 C190 P13670 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 19209 T2749 C190 P13670 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 19209 T2749 C190 P13670 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:13661/collection1/. core=collection1
[junit4:junit4]   2> 19209 T2749 C190 P13670 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19211 T2705 C191 P13661 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 19221 T2709 C191 P13661 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 19224 T2709 C191 P13661 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61f6247e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b339fce),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61f6247e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b339fce),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 19224 T2709 C191 P13661 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 19225 T2709 C191 P13661 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 19225 T2709 C191 P13661 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4]   2> 19226 T2749 C190 P13670 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 19226 T2749 C190 P13670 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 19228 T2707 C191 P13661 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 19228 T2749 C190 P13670 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4]   2> 19229 T2749 C190 P13670 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 19229 T2749 C190 P13670 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 19231 T2707 C191 P13661 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=0 
[junit4:junit4]   2> 19231 T2749 C190 P13670 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 19232 T2749 C190 P13670 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty3/index.20130314143630733 forceNew: false
[junit4:junit4]   2> 19232 T2749 C190 P13670 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@655f676b lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d0630d7) fullCopy=false
[junit4:junit4]   2> 19235 T2707 C191 P13661 REQ /replication {file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
[junit4:junit4]   2> 19236 T2749 C190 P13670 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 19246 T2749 C190 P13670 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 19246 T2749 C190 P13670 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 19299 T2749 C190 P13670 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12c6123d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e0f2b80),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12c6123d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e0f2b80),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 19300 T2749 C190 P13670 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 19300 T2749 C190 P13670 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 19300 T2749 C190 P13670 oass.SolrIndexSearcher.<init> Opening Searcher@240f2262 main
[junit4:junit4]   2> 19302 T2748 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@240f2262 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4]   2> 19302 T2749 C190 P13670 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@655f676b lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d0630d7)
[junit4:junit4]   2> 19303 T2749 C190 P13670 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 19303 T2749 C190 P13670 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 19303 T2749 C190 P13670 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 19303 T2749 C190 P13670 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19309 T2749 C190 P13670 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 19337 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19338 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13691_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13691"}
[junit4:junit4]   2> 19355 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13670_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13670"}
[junit4:junit4]   2> 19369 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19369 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19369 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19369 T2763 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19369 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19369 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19369 T2746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19784 T2708 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=5&state=recovering&nodeName=127.0.0.1:13691_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001 
[junit4:junit4]   2> 20192 T2671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13698_
[junit4:junit4]   2> 20194 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f000e type:delete cxid:0xd zxid:0xb5 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:13698_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:13698_
[junit4:junit4]   2> 20195 T2671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:13698_
[junit4:junit4]   2> 20205 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20205 T2746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20205 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20206 T2782 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 20206 T2763 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 20206 T2693 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 20206 T2763 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 20206 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 20206 T2729 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 20206 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 20207 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 20208 T2746 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 20208 T2700 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 20208 T2713 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 20215 T2785 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363293390284/collection1
[junit4:junit4]   2> 20216 T2785 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 20216 T2785 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 20216 T2785 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 20218 T2785 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363293390284/collection1/'
[junit4:junit4]   2> 20220 T2785 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty5-1363293390284/collection1/lib/README' to classloader
[junit4:junit4]   2> 20221 T2785 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty5-1363293390284/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 20285 T2785 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 20357 T2785 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 20458 T2785 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 20464 T2785 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 21205 T2785 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 21219 T2785 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 21224 T2785 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 21238 T2785 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 21244 T2785 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 21249 T2785 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 21251 T2785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 21251 T2785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 21252 T2785 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 21253 T2785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 21253 T2785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 21253 T2785 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 21254 T2785 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363293390284/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty5/
[junit4:junit4]   2> 21254 T2785 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2658dd2d
[junit4:junit4]   2> 21254 T2785 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 21255 T2785 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty5 forceNew: false
[junit4:junit4]   2> 21255 T2785 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty5/index/
[junit4:junit4]   2> 21255 T2785 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 21256 T2785 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty5/index forceNew: false
[junit4:junit4]   2> 21259 T2785 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2f26ac7c lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c5838aa),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21259 T2785 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 21261 T2785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 21262 T2785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 21262 T2785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 21263 T2785 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 21264 T2785 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 21264 T2785 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 21264 T2785 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 21265 T2785 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 21265 T2785 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 21274 T2785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 21280 T2785 oass.SolrIndexSearcher.<init> Opening Searcher@74e654ce main
[junit4:junit4]   2> 21281 T2785 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty5/tlog
[junit4:junit4]   2> 21282 T2785 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 21282 T2785 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 21287 T2786 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@74e654ce main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 21290 T2785 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 21290 T2785 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2>  C192_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:13691_, base_url=http://127.0.0.1:13691}
[junit4:junit4]   2> 21785 T2768 C192 P13691 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:13661/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 21785 T2768 C192 P13691 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:13691 START replicas=[http://127.0.0.1:13661/collection1/] nUpdates=100
[junit4:junit4]   2> 21786 T2768 C192 P13691 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 21786 T2768 C192 P13691 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 21786 T2768 C192 P13691 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 21786 T2768 C192 P13691 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 21786 T2768 C192 P13691 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 21786 T2768 C192 P13691 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:13661/collection1/. core=collection1
[junit4:junit4]   2> 21787 T2768 C192 P13691 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 21788 T2705 C191 P13661 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 21805 T2703 C191 P13661 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 21806 T2703 C191 P13661 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61f6247e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b339fce),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61f6247e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b339fce),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61f6247e lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b339fce),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 21807 T2703 C191 P13661 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 21807 T2703 C191 P13661 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 21807 T2703 C191 P13661 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 2
[junit4:junit4]   2> 21808 T2768 C192 P13691 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21808 T2768 C192 P13691 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 21811 T2707 C191 P13661 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 21811 T2768 C192 P13691 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4]   2> 21812 T2768 C192 P13691 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 21812 T2768 C192 P13691 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 21814 T2707 C191 P13661 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=0 
[junit4:junit4]   2> 21814 T2768 C192 P13691 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 21815 T2768 C192 P13691 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363293371508/jetty4/index.20130314143633316 forceNew: false
[junit4:junit4]   2> 21816 T2768 C192 P13691 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a825091 lockFactory=org.apache.lucene.store.NativeFSLockFactory@bd6e963) fullCopy=false
[junit4:junit4]   2> 21818 T2707 C191 P13661 REQ /replication {file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
[junit4:junit4]   2> 21819 T2768 C192 P13691 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 21829 T2768 C192 P13691 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 21829 T2768 C192 P13691 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 21881 T2768 C192 P13691 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@21917d40 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4841c79a),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@21917d40 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4841c79a),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 21882 T2768 C192 P13691 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 21882 T2768 C192 P13691 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 21882 T2768 C192 P13691 oass.SolrIndexSearcher.<init> Opening Searcher@4f8e835c main
[junit4:junit4]   2> 21883 T2767 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4f8e835c main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4]   2> 21884 T2768 C192 P13691 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a825091 lockFactory=org.apache.lucene.store.NativeFSLockFactory@bd6e963)
[junit4:junit4]   2> 21884 T2768 C192 P13691 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 21884 T2768 C192 P13691 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 21884 T2768 C192 P13691 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 21884 T2768 C192 P13691 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 21886 T2768 C192 P13691 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 22377 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22378 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"6",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13698_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13698"}
[junit4:junit4]   2> 22379 T2694 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 22379 T2694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 22392 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13691_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13691"}
[junit4:junit4]   2> 22412 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 22412 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 22412 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 22412 T2746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 22412 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 22412 T2763 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 22412 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23298 T2785 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 23298 T2785 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:13698 collection:collection1 shard:shard1
[junit4:junit4]   2> 23302 T2785 oasc.ZkController.register We are http://127.0.0.1:13698/collection1/ and leader is http://127.0.0.1:13661/collection1/
[junit4:junit4]   2> 23302 T2785 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:13698
[junit4:junit4]   2> 23302 T2785 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 23302 T2785 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C193 name=collection1 org.apache.solr.core.SolrCore@7fea7578 url=http://127.0.0.1:13698/collection1 node=127.0.0.1:13698_ C193_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:13698_, base_url=http://127.0.0.1:13698}
[junit4:junit4]   2> 23303 T2789 C193 P13698 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 23303 T2789 C193 P13698 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 23303 T2789 C193 P13698 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 23303 T2789 C193 P13698 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 23304 T2785 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23305 T2789 C193 P13698 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 23305 T2671 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 23306 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 23306 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 23578 T2671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 23581 T2671 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13704
[junit4:junit4]   2> 23581 T2671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 23582 T2671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 23582 T2671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty6-1363293394819
[junit4:junit4]   2> 23583 T2671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty6-1363293394819/solr.xml
[junit4:junit4]   2> 23583 T2671 oasc.CoreContainer.<init> New CoreContainer 9547283
[junit4:junit4]   2> 23584 T2671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty6-1363293394819/'
[junit4:junit4]   2> 23584 T2671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty6-1363293394819/'
[junit4:junit4]   2> 23657 T2671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 23658 T2671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 23658 T2671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 23659 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 23659 T2671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 23660 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 23660 T2671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 23661 T2671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 23661 T2671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 23661 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 23685 T2671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 23708 T2671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:13653/solr
[junit4:junit4]   2> 23709 T2671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 23709 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3e8a5777
[junit4:junit4]   2> 23710 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 23711 T2800 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 23712 T2800 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 23712 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13705
[junit4:junit4]   2> 23712 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13705
[junit4:junit4]   2> 23714 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f000f with negotiated timeout 20000 for client /140.211.11.196:13705
[junit4:junit4]   2> 23714 T2800 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f000f, negotiated timeout = 20000
[junit4:junit4]   2> 23714 T2801 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e8a5777 name:ZooKeeperConnection Watcher:127.0.0.1:13653 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 23714 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 23716 T2676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6a9dfc7f000f
[junit4:junit4]   2> 23717 T2801 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 23717 T2673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13705 which had sessionid 0x13d6a9dfc7f000f
[junit4:junit4]   2> 23717 T2671 oaz.ZooKeeper.close Session: 0x13d6a9dfc7f000f closed
[junit4:junit4]   2> 23718 T2671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 23728 T2671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:13653/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6eb481a2
[junit4:junit4]   2> 23730 T2671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 23730 T2802 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:13653. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 23731 T2802 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:13653, initiating session
[junit4:junit4]   2> 23731 T2673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13706
[junit4:junit4]   2> 23731 T2673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13706
[junit4:junit4]   2> 23733 T2675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6a9dfc7f0010 with negotiated timeout 20000 for client /140.211.11.196:13706
[junit4:junit4]   2> 23733 T2802 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:13653, sessionid = 0x13d6a9dfc7f0010, negotiated timeout = 20000
[junit4:junit4]   2> 23733 T2803 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6eb481a2 name:ZooKeeperConnection Watcher:127.0.0.1:13653/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 23733 T2671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 23734 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0010 type:create cxid:0x1 zxid:0xc6 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 23736 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0010 type:create cxid:0x2 zxid:0xc7 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 23744 T2671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 23932 T2694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23933 T2694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"6",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:13698_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:13698"}
[junit4:junit4]   2> 23948 T2713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23948 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23948 T2803 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23948 T2700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23948 T2729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23948 T2746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23948 T2763 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23948 T2693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 24318 T2706 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=6&state=recovering&nodeName=127.0.0.1:13698_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1000 
[junit4:junit4]   2> 24747 T2671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13704_
[junit4:junit4]   2> 24748 T2676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6a9dfc7f0010 type:delete cxid:0xd zxid:0xcc txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:13704_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:13

[...truncated too long message...]

ver.shutdown shutting down
[junit4:junit4]   2> 641176 T2671 oazs.SessionTrackerImpl.shutdown Shutting down
[junit4:junit4]   2> 641176 T2671 oazs.PrepRequestProcessor.shutdown Shutting down
[junit4:junit4]   2> 641177 T2671 oazs.SyncRequestProcessor.shutdown Shutting down
[junit4:junit4]   2> 641177 T2671 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=7ADAB6D4D7D0DD92 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=en_GB -Dtests.timezone=America/Costa_Rica -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE  641s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: shard1 is not consistent.  Got 305 from http://127.0.0.1:13698/collection1lastClient and got 5 from http://127.0.0.1:13704/collection1
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([7ADAB6D4D7D0DD92:FB3C38CCA08FBDAE]:0)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:963)
[junit4:junit4]    > 	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:235)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:616)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]   2> 641217 T2671 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 641222 T2670 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4]   2> 641500 T2674 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4]   2> NOTE: test params are: codec=Appending, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=en_GB, timezone=America/Costa_Rica
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Sun Microsystems Inc. 1.6.0_32 (64-bit)/cpus=16,threads=1,free=230976968,total=401670144
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SOLR749Test, SoftAutoCommitTest, SimpleFacetsTest, IndexSchemaTest, TestFieldTypeCollectionResource, TestStressRecovery, BinaryUpdateRequestHandlerTest, TestSearchPerf, TestSolrIndexConfig, AlternateDirectoryTest, ClusterStateUpdateTest, UnloadDistributedZkTest, JsonLoaderTest, TestLRUCache, TestLFUCache, BasicDistributedZkTest, PolyFieldTest, TestGroupingSearch, TestPHPSerializedResponseWriter, TermsComponentTest, ZkCLITest, TestPropInjectDefaults, ReturnFieldsTest, SuggesterFSTTest, ZkControllerTest, TestRandomDVFaceting, DocValuesTest, SolrIndexSplitterTest, StatelessScriptUpdateProcessorFactoryTest, MBeansHandlerTest, SpellingQueryConverterTest, ExternalFileFieldSortTest, DistributedQueryElevationComponentTest, LegacyHTMLStripCharFilterTest, LoggingHandlerTest, TestSolrDeletionPolicy1, QueryParsingTest, LeaderElectionTest, TestSolrDeletionPolicy2, MoreLikeThisHandlerTest, TestTrie, TestStressVersions, PreAnalyzedFieldTest, ResourceLoaderTest, FileBasedSpellCheckerTest, LukeRequestHandlerTest, SpatialFilterTest, PeerSyncTest, FieldMutatingUpdateProcessorTest, BadComponentTest, TestPhraseSuggestions, TestCollationKeyRangeQueries, SynonymTokenizerTest, SystemInfoHandlerTest, SuggesterTest, TestPluginEnable, QueryElevationComponentTest, SearchHandlerTest, TestFastWriter, BasicFunctionalityTest, ShowFileRequestHandlerTest, TestFunctionQuery, TestSolrCoreProperties, TermVectorComponentTest, TestReplicationHandler, TestJoin, CollectionsAPIDistributedZkTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 641.76s, 1 test, 1 failure <<< FAILURES!

[...truncated 382 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:388: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:877: There were test failures: 273 suites, 1147 tests, 1 failure, 25 ignored (6 assumptions)

Total time: 95 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message