lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3810 - Failure
Date Thu, 14 Mar 2013 11:47:07 GMT
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3810/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch

Error Message:
No registered leader was found, collection:collection1 slice:shard1

Stack Trace:
org.apache.solr.common.SolrException: No registered leader was found, collection:collection1 slice:shard1
	at __randomizedtesting.SeedInfo.seed([BAA74CC33B8DA7A2:3B41C2DB4CD2C79E]:0)
	at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:484)
	at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:291)
	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:116)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9240 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4]   2> 0 T1586 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 5 T1586 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1363260901452
[junit4:junit4]   2> 6 T1586 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 6 T1587 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 9 T1587 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 9 T1587 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 9 T1587 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 9 T1587 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 10 T1587 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 107 T1586 oasc.ZkTestServer.run start zk server on port:44540
[junit4:junit4]   2> 107 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4d002893
[junit4:junit4]   2> 109 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T1592 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 109 T1592 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 109 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63873
[junit4:junit4]   2> 110 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63873
[junit4:junit4]   2> 110 T1590 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 112 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae88510000 with negotiated timeout 10000 for client /140.211.11.196:63873
[junit4:junit4]   2> 112 T1592 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae88510000, negotiated timeout = 10000
[junit4:junit4]   2> 113 T1593 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d002893 name:ZooKeeperConnection Watcher:127.0.0.1:44540 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 113 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 113 T1586 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 116 T1591 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d68ae88510000
[junit4:junit4]   2> 117 T1593 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 117 T1588 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:63873 which had sessionid 0x13d68ae88510000
[junit4:junit4]   2> 117 T1586 oaz.ZooKeeper.close Session: 0x13d68ae88510000 closed
[junit4:junit4]   2> 117 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1758541
[junit4:junit4]   2> 118 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 118 T1594 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 118 T1594 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 119 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13195
[junit4:junit4]   2> 119 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13195
[junit4:junit4]   2> 126 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae88510001 with negotiated timeout 10000 for client /140.211.11.196:13195
[junit4:junit4]   2> 126 T1594 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae88510001, negotiated timeout = 10000
[junit4:junit4]   2> 126 T1595 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1758541 name:ZooKeeperConnection Watcher:127.0.0.1:44540/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 126 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 127 T1586 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 130 T1586 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 132 T1586 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 135 T1586 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 137 T1586 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 138 T1586 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 149 T1586 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 149 T1586 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 252 T1586 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 253 T1586 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 256 T1586 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 257 T1586 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 266 T1586 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 267 T1586 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 270 T1586 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 271 T1586 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 274 T1586 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 275 T1586 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 278 T1586 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 279 T1586 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 282 T1586 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 282 T1586 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 285 T1591 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d68ae88510001
[junit4:junit4]   2> 293 T1595 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 293 T1588 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13195 which had sessionid 0x13d68ae88510001
[junit4:junit4]   2> 293 T1586 oaz.ZooKeeper.close Session: 0x13d68ae88510001 closed
[junit4:junit4]   2> 476 T1586 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 480 T1586 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57771
[junit4:junit4]   2> 481 T1586 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 481 T1586 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 482 T1586 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1363260901741
[junit4:junit4]   2> 482 T1586 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1363260901741/solr.xml
[junit4:junit4]   2> 482 T1586 oasc.CoreContainer.<init> New CoreContainer 1868157541
[junit4:junit4]   2> 483 T1586 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1363260901741/'
[junit4:junit4]   2> 483 T1586 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1363260901741/'
[junit4:junit4]   2> 534 T1586 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 535 T1586 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 535 T1586 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 535 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 536 T1586 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 536 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 536 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 537 T1586 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 537 T1586 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 537 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 553 T1586 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 588 T1586 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:44540/solr
[junit4:junit4]   2> 589 T1586 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 589 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3905f6f3
[junit4:junit4]   2> 590 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 591 T1605 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 591 T1605 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 592 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:44362
[junit4:junit4]   2> 592 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:44362
[junit4:junit4]   2> 594 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae88510002 with negotiated timeout 20000 for client /140.211.11.196:44362
[junit4:junit4]   2> 594 T1605 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae88510002, negotiated timeout = 20000
[junit4:junit4]   2> 594 T1606 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3905f6f3 name:ZooKeeperConnection Watcher:127.0.0.1:44540 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 594 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 596 T1591 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d68ae88510002
[junit4:junit4]   2> 597 T1606 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 597 T1588 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:44362 which had sessionid 0x13d68ae88510002
[junit4:junit4]   2> 597 T1586 oaz.ZooKeeper.close Session: 0x13d68ae88510002 closed
[junit4:junit4]   2> 597 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 605 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@64bd79dd
[junit4:junit4]   2> 606 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 606 T1607 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 606 T1607 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 607 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:43833
[junit4:junit4]   2> 607 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:43833
[junit4:junit4]   2> 608 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae88510003 with negotiated timeout 20000 for client /140.211.11.196:43833
[junit4:junit4]   2> 608 T1607 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae88510003, negotiated timeout = 20000
[junit4:junit4]   2> 608 T1608 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64bd79dd name:ZooKeeperConnection Watcher:127.0.0.1:44540/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 608 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 610 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 612 T1586 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 620 T1586 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57771_
[junit4:junit4]   2> 620 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57771_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57771_
[junit4:junit4]   2> 621 T1586 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57771_
[junit4:junit4]   2> 628 T1586 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 642 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 643 T1586 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 646 T1586 oasc.Overseer.start Overseer (id=89342666437885955-127.0.0.1:57771_-n_0000000000) starting
[junit4:junit4]   2> 646 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 654 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 655 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 661 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 663 T1610 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 663 T1586 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 674 T1586 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 676 T1586 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 678 T1609 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 681 T1611 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1363260901741/collection1
[junit4:junit4]   2> 681 T1611 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 681 T1611 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 682 T1611 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 683 T1611 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1363260901741/collection1/'
[junit4:junit4]   2> 684 T1611 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1363260901741/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 684 T1611 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1363260901741/collection1/lib/README' to classloader
[junit4:junit4]   2> 735 T1611 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 801 T1611 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 902 T1611 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 909 T1611 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1602 T1611 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1614 T1611 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1619 T1611 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1635 T1611 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1641 T1611 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1646 T1611 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1647 T1611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1648 T1611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 1648 T1611 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1649 T1611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1649 T1611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 1650 T1611 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1650 T1611 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1363260901741/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/control/data/
[junit4:junit4]   2> 1650 T1611 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b99d741
[junit4:junit4]   2> 1651 T1611 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1652 T1611 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/control/data forceNew: false
[junit4:junit4]   2> 1652 T1611 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/control/data/index/
[junit4:junit4]   2> 1652 T1611 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1653 T1611 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/control/data/index forceNew: false
[junit4:junit4]   2> 1657 T1611 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a7de20b lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e665d45),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1657 T1611 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1660 T1611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1660 T1611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1661 T1611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1661 T1611 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1662 T1611 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1662 T1611 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1662 T1611 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1663 T1611 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1663 T1611 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1678 T1611 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1685 T1611 oass.SolrIndexSearcher.<init> Opening Searcher@9e5df3b main
[junit4:junit4]   2> 1686 T1611 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/control/data/tlog
[junit4:junit4]   2> 1687 T1611 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1687 T1611 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1690 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9e5df3b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1692 T1611 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1692 T1611 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1693 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:create cxid:0x55 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2182 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2183 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57771_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57771"}
[junit4:junit4]   2> 2183 T1609 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2183 T1609 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2184 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:create cxid:0x5d zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2196 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2707 T1611 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2707 T1611 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57771 collection:control_collection shard:shard1
[junit4:junit4]   2> 2708 T1611 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2719 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:delete cxid:0x77 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2720 T1611 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2720 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:create cxid:0x78 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2729 T1611 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2730 T1611 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2730 T1611 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57771/collection1/
[junit4:junit4]   2> 2730 T1611 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2730 T1611 oasc.SyncStrategy.syncToMe http://127.0.0.1:57771/collection1/ has no replicas
[junit4:junit4]   2> 2731 T1611 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57771/collection1/
[junit4:junit4]   2> 2731 T1611 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2736 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510003 type:create cxid:0x82 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3702 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3721 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3759 T1611 oasc.ZkController.register We are http://127.0.0.1:57771/collection1/ and leader is http://127.0.0.1:57771/collection1/
[junit4:junit4]   2> 3759 T1611 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57771
[junit4:junit4]   2> 3759 T1611 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3759 T1611 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3760 T1611 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3762 T1611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3763 T1586 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3764 T1586 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3764 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3779 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3780 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7c222504
[junit4:junit4]   2> 3781 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3782 T1614 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3782 T1614 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 3783 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58242
[junit4:junit4]   2> 3783 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58242
[junit4:junit4]   2> 3784 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae88510004 with negotiated timeout 10000 for client /140.211.11.196:58242
[junit4:junit4]   2> 3784 T1614 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae88510004, negotiated timeout = 10000
[junit4:junit4]   2> 3785 T1615 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c222504 name:ZooKeeperConnection Watcher:127.0.0.1:44540/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3785 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3787 T1586 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3789 T1586 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3998 T1586 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4001 T1586 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:12179
[junit4:junit4]   2> 4002 T1586 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4002 T1586 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4003 T1586 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236
[junit4:junit4]   2> 4003 T1586 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/solr.xml
[junit4:junit4]   2> 4004 T1586 oasc.CoreContainer.<init> New CoreContainer 1962423853
[junit4:junit4]   2> 4005 T1586 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/'
[junit4:junit4]   2> 4005 T1586 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/'
[junit4:junit4]   2> 4074 T1586 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 4074 T1586 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4075 T1586 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4075 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4076 T1586 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4076 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4077 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4077 T1586 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4078 T1586 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4078 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4099 T1586 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4120 T1586 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:44540/solr
[junit4:junit4]   2> 4120 T1586 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4121 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7a50add3
[junit4:junit4]   2> 4122 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4123 T1625 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4123 T1625 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 4124 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57000
[junit4:junit4]   2> 4124 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57000
[junit4:junit4]   2> 4126 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae88510005 with negotiated timeout 20000 for client /140.211.11.196:57000
[junit4:junit4]   2> 4126 T1625 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae88510005, negotiated timeout = 20000
[junit4:junit4]   2> 4126 T1626 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a50add3 name:ZooKeeperConnection Watcher:127.0.0.1:44540 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4127 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4128 T1591 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d68ae88510005
[junit4:junit4]   2> 4129 T1626 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4129 T1588 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57000 which had sessionid 0x13d68ae88510005
[junit4:junit4]   2> 4129 T1586 oaz.ZooKeeper.close Session: 0x13d68ae88510005 closed
[junit4:junit4]   2> 4130 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 4141 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5c43152c
[junit4:junit4]   2> 4142 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4142 T1627 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4143 T1627 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 4143 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:62654
[junit4:junit4]   2> 4144 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:62654
[junit4:junit4]   2> 4145 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae88510006 with negotiated timeout 20000 for client /140.211.11.196:62654
[junit4:junit4]   2> 4145 T1627 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae88510006, negotiated timeout = 20000
[junit4:junit4]   2> 4145 T1628 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c43152c name:ZooKeeperConnection Watcher:127.0.0.1:44540/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4146 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4147 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510006 type:create cxid:0x1 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4148 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510006 type:create cxid:0x2 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4157 T1586 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5161 T1586 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:12179_
[junit4:junit4]   2> 5163 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510006 type:delete cxid:0xb zxid:0x4c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:12179_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:12179_
[junit4:junit4]   2> 5189 T1586 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:12179_
[junit4:junit4]   2> 5193 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 5194 T1615 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5194 T1628 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5196 T1608 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5204 T1629 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/collection1
[junit4:junit4]   2> 5204 T1629 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5205 T1629 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5205 T1629 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5207 T1629 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/collection1/'
[junit4:junit4]   2> 5209 T1629 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5210 T1629 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/collection1/lib/README' to classloader
[junit4:junit4]   2> 5241 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5242 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57771_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57771"}
[junit4:junit4]   2> 5255 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5255 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5255 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5267 T1629 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 5355 T1629 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5456 T1629 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5463 T1629 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6233 T1629 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6246 T1629 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6251 T1629 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6264 T1629 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6270 T1629 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6275 T1629 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6276 T1629 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 6276 T1629 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 6277 T1629 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6278 T1629 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 6278 T1629 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 6278 T1629 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 6278 T1629 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty1/
[junit4:junit4]   2> 6279 T1629 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b99d741
[junit4:junit4]   2> 6279 T1629 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 6280 T1629 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty1 forceNew: false
[junit4:junit4]   2> 6280 T1629 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty1/index/
[junit4:junit4]   2> 6280 T1629 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6280 T1629 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty1/index forceNew: false
[junit4:junit4]   2> 6283 T1629 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12a31785 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a7eda65),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6283 T1629 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 6285 T1629 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6285 T1629 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6285 T1629 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6286 T1629 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6286 T1629 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6287 T1629 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6287 T1629 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6287 T1629 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6287 T1629 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6298 T1629 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6303 T1629 oass.SolrIndexSearcher.<init> Opening Searcher@6c495a9 main
[junit4:junit4]   2> 6303 T1629 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty1/tlog
[junit4:junit4]   2> 6304 T1629 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6304 T1629 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6307 T1630 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6c495a9 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6308 T1629 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 6308 T1629 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6760 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6761 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:12179_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:12179"}
[junit4:junit4]   2> 6761 T1609 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 6761 T1609 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6777 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6777 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6777 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7312 T1629 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7312 T1629 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:12179 collection:collection1 shard:shard1
[junit4:junit4]   2> 7313 T1629 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7332 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510006 type:delete cxid:0x48 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7332 T1629 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7333 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510006 type:create cxid:0x49 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7342 T1629 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7342 T1629 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7342 T1629 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:12179/collection1/
[junit4:junit4]   2> 7342 T1629 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7342 T1629 oasc.SyncStrategy.syncToMe http://127.0.0.1:12179/collection1/ has no replicas
[junit4:junit4]   2> 7342 T1629 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:12179/collection1/
[junit4:junit4]   2> 7343 T1629 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7354 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510006 type:create cxid:0x53 zxid:0x61 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8290 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8320 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8320 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8320 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8325 T1629 oasc.ZkController.register We are http://127.0.0.1:12179/collection1/ and leader is http://127.0.0.1:12179/collection1/
[junit4:junit4]   2> 8325 T1629 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:12179
[junit4:junit4]   2> 8325 T1629 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8325 T1629 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8326 T1629 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8329 T1629 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8330 T1586 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 8330 T1586 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8331 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8471 T1586 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8474 T1586 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22200
[junit4:junit4]   2> 8474 T1586 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8475 T1586 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8475 T1586 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1363260909789
[junit4:junit4]   2> 8475 T1586 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1363260909789/solr.xml
[junit4:junit4]   2> 8476 T1586 oasc.CoreContainer.<init> New CoreContainer 750106771
[junit4:junit4]   2> 8476 T1586 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1363260909789/'
[junit4:junit4]   2> 8477 T1586 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1363260909789/'
[junit4:junit4]   2> 8520 T1586 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 8520 T1586 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8521 T1586 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 8521 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8521 T1586 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8522 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8522 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8522 T1586 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8522 T1586 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8523 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 8538 T1586 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8552 T1586 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:44540/solr
[junit4:junit4]   2> 8552 T1586 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 8553 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5725f019
[junit4:junit4]   2> 8553 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8554 T1641 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8555 T1641 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 8555 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22202
[junit4:junit4]   2> 8555 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22202
[junit4:junit4]   2> 8557 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae88510007 with negotiated timeout 20000 for client /140.211.11.196:22202
[junit4:junit4]   2> 8557 T1641 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae88510007, negotiated timeout = 20000
[junit4:junit4]   2> 8558 T1642 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5725f019 name:ZooKeeperConnection Watcher:127.0.0.1:44540 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8558 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8559 T1591 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d68ae88510007
[junit4:junit4]   2> 8571 T1642 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 8571 T1588 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22202 which had sessionid 0x13d68ae88510007
[junit4:junit4]   2> 8571 T1586 oaz.ZooKeeper.close Session: 0x13d68ae88510007 closed
[junit4:junit4]   2> 8572 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 8584 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@44c309a6
[junit4:junit4]   2> 8585 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8586 T1643 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8586 T1643 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 8586 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22206
[junit4:junit4]   2> 8587 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22206
[junit4:junit4]   2> 8588 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae88510008 with negotiated timeout 20000 for client /140.211.11.196:22206
[junit4:junit4]   2> 8588 T1643 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae88510008, negotiated timeout = 20000
[junit4:junit4]   2> 8588 T1644 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44c309a6 name:ZooKeeperConnection Watcher:127.0.0.1:44540/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8589 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8591 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510008 type:create cxid:0x1 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8592 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510008 type:create cxid:0x2 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8597 T1586 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9605 T1586 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22200_
[junit4:junit4]   2> 9607 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510008 type:delete cxid:0xb zxid:0x70 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22200_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22200_
[junit4:junit4]   2> 9612 T1586 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22200_
[junit4:junit4]   2> 9616 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9616 T1644 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9616 T1615 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9616 T1628 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9616 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9617 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9618 T1608 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9639 T1645 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1363260909789/collection1
[junit4:junit4]   2> 9639 T1645 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9639 T1645 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9640 T1645 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9641 T1645 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1363260909789/collection1/'
[junit4:junit4]   2> 9642 T1645 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1363260909789/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9643 T1645 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1363260909789/collection1/lib/README' to classloader
[junit4:junit4]   2> 9687 T1645 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 9748 T1645 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9826 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9827 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:12179_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:12179"}
[junit4:junit4]   2> 9842 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9842 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9842 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9850 T1645 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9850 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9857 T1645 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10665 T1645 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10679 T1645 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10682 T1645 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10704 T1645 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10711 T1645 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10716 T1645 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10718 T1645 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 10719 T1645 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 10719 T1645 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10720 T1645 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 10720 T1645 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 10721 T1645 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 10721 T1645 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1363260909789/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty2/
[junit4:junit4]   2> 10722 T1645 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b99d741
[junit4:junit4]   2> 10722 T1645 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 10724 T1645 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty2 forceNew: false
[junit4:junit4]   2> 10724 T1645 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty2/index/
[junit4:junit4]   2> 10724 T1645 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10724 T1645 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty2/index forceNew: false
[junit4:junit4]   2> 10728 T1645 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@170153a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dff8a4e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10729 T1645 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 10732 T1645 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10732 T1645 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10733 T1645 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10734 T1645 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10734 T1645 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10734 T1645 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10735 T1645 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10736 T1645 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10736 T1645 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10751 T1645 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10759 T1645 oass.SolrIndexSearcher.<init> Opening Searcher@6892e965 main
[junit4:junit4]   2> 10760 T1645 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty2/tlog
[junit4:junit4]   2> 10761 T1645 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10761 T1645 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10767 T1646 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6892e965 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10769 T1645 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 10770 T1645 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11374 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11388 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22200_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22200"}
[junit4:junit4]   2> 11388 T1609 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 11388 T1609 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 11441 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11446 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11448 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11458 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11775 T1645 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 11775 T1645 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:22200 collection:collection1 shard:shard2
[junit4:junit4]   2> 11776 T1645 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 11794 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510008 type:delete cxid:0x47 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 11795 T1645 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 11796 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510008 type:create cxid:0x48 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11798 T1645 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 11799 T1645 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 11799 T1645 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:22200/collection1/
[junit4:junit4]   2> 11799 T1645 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 11799 T1645 oasc.SyncStrategy.syncToMe http://127.0.0.1:22200/collection1/ has no replicas
[junit4:junit4]   2> 11800 T1645 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:22200/collection1/
[junit4:junit4]   2> 11800 T1645 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 11804 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510008 type:create cxid:0x51 zxid:0x83 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12966 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12989 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12989 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12990 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12989 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13036 T1645 oasc.ZkController.register We are http://127.0.0.1:22200/collection1/ and leader is http://127.0.0.1:22200/collection1/
[junit4:junit4]   2> 13036 T1645 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22200
[junit4:junit4]   2> 13037 T1645 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 13037 T1645 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 13037 T1645 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13040 T1645 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13042 T1586 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 13043 T1586 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13044 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13479 T1586 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 13482 T1586 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22220
[junit4:junit4]   2> 13483 T1586 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13485 T1586 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13485 T1586 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1363260914514
[junit4:junit4]   2> 13486 T1586 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1363260914514/solr.xml
[junit4:junit4]   2> 13487 T1586 oasc.CoreContainer.<init> New CoreContainer 850131145
[junit4:junit4]   2> 13488 T1586 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1363260914514/'
[junit4:junit4]   2> 13488 T1586 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1363260914514/'
[junit4:junit4]   2> 13737 T1586 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 13738 T1586 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13739 T1586 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 13740 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13740 T1586 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13741 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13742 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13743 T1586 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13743 T1586 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13744 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 13784 T1586 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13816 T1586 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:44540/solr
[junit4:junit4]   2> 13817 T1586 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 13817 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4e6e977f
[junit4:junit4]   2> 13819 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13819 T1657 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13820 T1657 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 13820 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22221
[junit4:junit4]   2> 13821 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22221
[junit4:junit4]   2> 13823 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae88510009 with negotiated timeout 20000 for client /140.211.11.196:22221
[junit4:junit4]   2> 13824 T1657 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae88510009, negotiated timeout = 20000
[junit4:junit4]   2> 13824 T1658 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e6e977f name:ZooKeeperConnection Watcher:127.0.0.1:44540 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13824 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13826 T1591 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d68ae88510009
[junit4:junit4]   2> 13840 T1658 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 13840 T1588 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22221 which had sessionid 0x13d68ae88510009
[junit4:junit4]   2> 13840 T1586 oaz.ZooKeeper.close Session: 0x13d68ae88510009 closed
[junit4:junit4]   2> 13841 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 13854 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3a76ef83
[junit4:junit4]   2> 13856 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13857 T1659 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13858 T1659 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 13858 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22222
[junit4:junit4]   2> 13859 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22222
[junit4:junit4]   2> 13860 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae8851000a with negotiated timeout 20000 for client /140.211.11.196:22222
[junit4:junit4]   2> 13861 T1659 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae8851000a, negotiated timeout = 20000
[junit4:junit4]   2> 13861 T1660 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a76ef83 name:ZooKeeperConnection Watcher:127.0.0.1:44540/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13861 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13863 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae8851000a type:create cxid:0x1 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13867 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae8851000a type:create cxid:0x2 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13874 T1586 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 14507 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14509 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22200_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22200"}
[junit4:junit4]   2> 14529 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14529 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14529 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14529 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14529 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14889 T1586 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22220_
[junit4:junit4]   2> 14890 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae8851000a type:delete cxid:0xd zxid:0x96 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22220_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22220_
[junit4:junit4]   2> 14894 T1586 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22220_
[junit4:junit4]   2> 14903 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14903 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14903 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14905 T1615 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14905 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14906 T1628 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14906 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14909 T1608 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14909 T1660 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14912 T1644 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14920 T1661 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1363260914514/collection1
[junit4:junit4]   2> 14920 T1661 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 14921 T1661 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 14922 T1661 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 14924 T1661 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1363260914514/collection1/'
[junit4:junit4]   2> 14925 T1661 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1363260914514/collection1/lib/README' to classloader
[junit4:junit4]   2> 14926 T1661 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1363260914514/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 14997 T1661 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 15076 T1661 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 15178 T1661 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 15185 T1661 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 17059 T1661 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 17073 T1661 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 17076 T1661 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 17094 T1661 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17108 T1661 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17114 T1661 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17116 T1661 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 17116 T1661 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 17116 T1661 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17118 T1661 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 17118 T1661 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 17118 T1661 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 17119 T1661 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1363260914514/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty3/
[junit4:junit4]   2> 17119 T1661 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b99d741
[junit4:junit4]   2> 17120 T1661 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 17121 T1661 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty3 forceNew: false
[junit4:junit4]   2> 17121 T1661 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty3/index/
[junit4:junit4]   2> 17121 T1661 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 17122 T1661 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty3/index forceNew: false
[junit4:junit4]   2> 17125 T1661 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cfd75e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d024f4f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 17125 T1661 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 17127 T1661 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 17128 T1661 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 17128 T1661 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 17129 T1661 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 17130 T1661 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 17130 T1661 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 17130 T1661 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 17130 T1661 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 17131 T1661 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 17141 T1661 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 17147 T1661 oass.SolrIndexSearcher.<init> Opening Searcher@4cede0e3 main
[junit4:junit4]   2> 17147 T1661 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty3/tlog
[junit4:junit4]   2> 17148 T1661 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 17148 T1661 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 17152 T1662 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4cede0e3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 17154 T1661 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 17155 T1661 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17545 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17546 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22220_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22220"}
[junit4:junit4]   2> 17547 T1609 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 17547 T1609 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 17562 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17563 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17563 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17562 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17563 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18158 T1661 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 18158 T1661 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:22220 collection:collection1 shard:shard1
[junit4:junit4]   2> 18163 T1661 oasc.ZkController.register We are http://127.0.0.1:22220/collection1/ and leader is http://127.0.0.1:12179/collection1/
[junit4:junit4]   2> 18163 T1661 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22220
[junit4:junit4]   2> 18163 T1661 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 18164 T1661 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C339 name=collection1 org.apache.solr.core.SolrCore@18580e53 url=http://127.0.0.1:22220/collection1 node=127.0.0.1:22220_ C339_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:22220_, base_url=http://127.0.0.1:22220}
[junit4:junit4]   2> 18164 T1663 C339 P22220 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 18165 T1661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18166 T1663 C339 P22220 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 18166 T1663 C339 P22220 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 18166 T1663 C339 P22220 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18167 T1586 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 18168 T1663 C339 P22220 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18168 T1586 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 18169 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18496 T1586 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 18506 T1586 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22224
[junit4:junit4]   2> 18509 T1586 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 18510 T1586 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 18511 T1586 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1363260919633
[junit4:junit4]   2> 18512 T1586 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1363260919633/solr.xml
[junit4:junit4]   2> 18512 T1586 oasc.CoreContainer.<init> New CoreContainer 139627035
[junit4:junit4]   2> 18514 T1586 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1363260919633/'
[junit4:junit4]   2> 18514 T1586 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1363260919633/'
[junit4:junit4]   2> 18649 T1586 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 18650 T1586 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 18651 T1586 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 18651 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 18652 T1586 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 18652 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 18653 T1586 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 18653 T1586 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 18654 T1586 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 18654 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 18678 T1586 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 18702 T1586 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:44540/solr
[junit4:junit4]   2> 18703 T1586 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 18704 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1dfb070
[junit4:junit4]   2> 18705 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18706 T1674 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 18707 T1674 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 18708 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22225
[junit4:junit4]   2> 18708 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22225
[junit4:junit4]   2> 18718 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae8851000b with negotiated timeout 20000 for client /140.211.11.196:22225
[junit4:junit4]   2> 18718 T1674 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae8851000b, negotiated timeout = 20000
[junit4:junit4]   2> 18718 T1675 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dfb070 name:ZooKeeperConnection Watcher:127.0.0.1:44540 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18719 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18721 T1591 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d68ae8851000b
[junit4:junit4]   2> 18723 T1675 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 18723 T1588 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22225 which had sessionid 0x13d68ae8851000b
[junit4:junit4]   2> 18723 T1586 oaz.ZooKeeper.close Session: 0x13d68ae8851000b closed
[junit4:junit4]   2> 18724 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 18740 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@2ea0167a
[junit4:junit4]   2> 18741 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18742 T1676 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 18743 T1676 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 18743 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22226
[junit4:junit4]   2> 18743 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22226
[junit4:junit4]   2> 18745 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae8851000c with negotiated timeout 20000 for client /140.211.11.196:22226
[junit4:junit4]   2> 18745 T1676 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae8851000c, negotiated timeout = 20000
[junit4:junit4]   2> 18746 T1677 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ea0167a name:ZooKeeperConnection Watcher:127.0.0.1:44540/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18746 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18747 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae8851000c type:create cxid:0x1 zxid:0xa3 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18749 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae8851000c type:create cxid:0x2 zxid:0xa4 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18752 T1586 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 19076 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19078 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22220_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22220"}
[junit4:junit4]   2> 19086 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19087 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19086 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19086 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19086 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19086 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19186 T1618 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:22220__collection1&state=recovering&nodeName=127.0.0.1:22220_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001 
[junit4:junit4]   2> 19759 T1586 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22224_
[junit4:junit4]   2> 19761 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae8851000c type:delete cxid:0xd zxid:0xa9 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22224_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22224_
[junit4:junit4]   2> 19762 T1586 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22224_
[junit4:junit4]   2> 19765 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19765 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19765 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19765 T1628 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 19765 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 19765 T1615 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 19766 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19766 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19767 T1660 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 19768 T1608 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 19768 T1677 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 19769 T1644 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 19774 T1678 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1363260919633/collection1
[junit4:junit4]   2> 19774 T1678 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 19775 T1678 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 19775 T1678 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 19777 T1678 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1363260919633/collection1/'
[junit4:junit4]   2> 19778 T1678 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1363260919633/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 19779 T1678 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1363260919633/collection1/lib/README' to classloader
[junit4:junit4]   2> 19834 T1678 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 19937 T1678 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 20039 T1678 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 20051 T1678 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 21111 T1678 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 21131 T1678 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 21135 T1678 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 21167 T1678 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 21180 T1678 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2>  C339_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:22220_, base_url=http://127.0.0.1:22220}
[junit4:junit4]   2> 21188 T1663 C339 P22220 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:12179/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 21188 T1663 C339 P22220 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:22220 START replicas=[http://127.0.0.1:12179/collection1/] nUpdates=100
[junit4:junit4]   2> 21189 T1663 C339 P22220 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 21191 T1663 C339 P22220 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 21191 T1663 C339 P22220 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 21191 T1663 C339 P22220 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 21192 T1663 C339 P22220 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 21192 T1663 C339 P22220 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:12179/collection1/. core=collection1
[junit4:junit4]   2> 21192 T1663 C339 P22220 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 21198 T1678 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 21200 T1678 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 21201 T1678 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 21202 T1678 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 21205 T1678 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 21205 T1678 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 21206 T1678 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 21206 T1678 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1363260919633/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4/
[junit4:junit4]   2> 21207 T1678 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b99d741
[junit4:junit4]   2> 21208 T1678 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> ASYNC  NEW_CORE C340 name=collection1 org.apache.solr.core.SolrCore@36b11c2c url=http://127.0.0.1:12179/collection1 node=127.0.0.1:12179_ C340_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:12179_, base_url=http://127.0.0.1:12179, leader=true}
[junit4:junit4]   2> 21208 T1619 C340 P12179 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 21209 T1678 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4 forceNew: false
[junit4:junit4]   2> 21210 T1678 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4/index/
[junit4:junit4]   2> 21210 T1678 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 21211 T1678 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4/index forceNew: false
[junit4:junit4]   2> 21212 T1620 C340 P12179 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 21215 T1678 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@516c2093 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc3c2ba),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21215 T1678 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 21216 T1620 C340 P12179 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12a31785 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a7eda65),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21217 T1620 C340 P12179 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 21218 T1620 C340 P12179 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12a31785 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a7eda65),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12a31785 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a7eda65),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21219 T1620 C340 P12179 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21219 T1678 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 21219 T1678 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 21219 T1620 C340 P12179 oass.SolrIndexSearcher.<init> Opening Searcher@6ed52e22 realtime
[junit4:junit4]   2> 21220 T1678 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 21220 T1620 C340 P12179 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 21221 T1620 C340 P12179 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4]   2> 21221 T1678 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 21222 T1678 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 21222 T1663 C339 P22220 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21222 T1678 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 21223 T1663 C339 P22220 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 21224 T1678 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 21225 T1678 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 21225 T1678 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 21227 T1621 C340 P12179 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21227 T1621 C340 P12179 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 21229 T1663 C339 P22220 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 21229 T1663 C339 P22220 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 21229 T1663 C339 P22220 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 21231 T1621 C340 P12179 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 21232 T1663 C339 P22220 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 21234 T1663 C339 P22220 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty3/index.20130314053522680 forceNew: false
[junit4:junit4]   2> 21234 T1663 C339 P22220 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@53d5b34b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7285cde6) fullCopy=false
[junit4:junit4]   2> 21237 T1621 C340 P12179 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 21240 T1663 C339 P22220 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 21241 T1678 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 21241 T1663 C339 P22220 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 21242 T1663 C339 P22220 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 21244 T1663 C339 P22220 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cfd75e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d024f4f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cfd75e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d024f4f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21244 T1663 C339 P22220 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21245 T1663 C339 P22220 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 21245 T1663 C339 P22220 oass.SolrIndexSearcher.<init> Opening Searcher@a411b1c main
[junit4:junit4]   2> 21246 T1662 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a411b1c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 21247 T1663 C339 P22220 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@53d5b34b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7285cde6)
[junit4:junit4]   2> 21247 T1663 C339 P22220 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 21247 T1663 C339 P22220 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 21248 T1663 C339 P22220 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 21248 T1663 C339 P22220 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 21249 T1678 oass.SolrIndexSearcher.<init> Opening Searcher@28d9d062 main
[junit4:junit4]   2> 21250 T1678 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4/tlog
[junit4:junit4]   2> 21250 T1663 C339 P22220 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 21251 T1678 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 21251 T1678 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 21256 T1681 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@28d9d062 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 21259 T1678 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 21259 T1678 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22131 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22133 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22220_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22220"}
[junit4:junit4]   2> 22144 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22224_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22224"}
[junit4:junit4]   2> 22145 T1609 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 22145 T1609 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 22156 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22156 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22157 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22157 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22157 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22157 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22263 T1678 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 22263 T1678 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:22224 collection:collection1 shard:shard2
[junit4:junit4]   2> 22267 T1678 oasc.ZkController.register We are http://127.0.0.1:22224/collection1/ and leader is http://127.0.0.1:22200/collection1/
[junit4:junit4]   2> 22267 T1678 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22224
[junit4:junit4]   2> 22268 T1678 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 22268 T1678 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C341 name=collection1 org.apache.solr.core.SolrCore@319e02e7 url=http://127.0.0.1:22224/collection1 node=127.0.0.1:22224_ C341_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:22224_, base_url=http://127.0.0.1:22224}
[junit4:junit4]   2> 22269 T1682 C341 P22224 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 22270 T1682 C341 P22224 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 22271 T1682 C341 P22224 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 22271 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22271 T1682 C341 P22224 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22273 T1586 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 22274 T1682 C341 P22224 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 22274 T1586 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 22274 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 22294 T1586 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22296 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 22312 T1622 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/onenodecollectioncore
[junit4:junit4]   2> 22312 T1622 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4]   2> 22320 T1622 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4]   2> 22321 T1622 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 22322 T1622 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 22323 T1622 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4]   2> 22329 T1622 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4]   2> 22332 T1622 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/onenodecollectioncore/'
[junit4:junit4]   2> 22487 T1622 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 22592 T1622 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 22694 T1622 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 22704 T1622 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 23678 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23683 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22224_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22224"}
[junit4:junit4]   2> 23700 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23700 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23701 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23700 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23700 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23701 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23894 T1622 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 23916 T1622 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 23925 T1622 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 23969 T1622 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 23976 T1622 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 23982 T1622 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 23983 T1622 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 23984 T1622 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 23984 T1622 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 23986 T1622 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 23987 T1622 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 23987 T1622 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 23988 T1622 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1363260905236/onenodecollectioncore/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1363260901452/onenodecollection/
[junit4:junit4]   2> 23988 T1622 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b99d741
[junit4:junit4]   2> 23989 T1622 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 23991 T1622 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1363260901452/onenodecollection forceNew: false
[junit4:junit4]   2> 23991 T1622 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1363260901452/onenodecollection/index/
[junit4:junit4]   2> 23992 T1622 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1363260901452/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 23993 T1622 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1363260901452/onenodecollection/index forceNew: false
[junit4:junit4]   2> 23997 T1622 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@159ce328 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a70b14c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 23998 T1622 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 24001 T1622 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 24002 T1622 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 24003 T1622 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 24004 T1622 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 24005 T1622 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 24006 T1622 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 24006 T1622 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 24007 T1622 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 24007 T1622 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 24044 T1622 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 24091 T1622 oass.SolrIndexSearcher.<init> Opening Searcher@626fbd05 main
[junit4:junit4]   2> 24092 T1622 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1363260901452/onenodecollection/tlog
[junit4:junit4]   2> 24094 T1622 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 24094 T1622 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 24099 T1684 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@626fbd05 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 24100 T1622 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4]   2> 24296 T1634 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:22224__collection1&state=recovering&nodeName=127.0.0.1:22224_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002 
[junit4:junit4]   2> 25213 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 25216 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":"none",
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"onenodecollectioncore",
[junit4:junit4]   2> 	  "collection":"onenodecollection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:12179_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:12179"}
[junit4:junit4]   2> 25216 T1609 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4]   2> 25217 T1609 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 25236 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 25236 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 25237 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 25236 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 25236 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 25237 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 26112 T1622 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4]   2> 26112 T1622 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:12179 collection:onenodecollection shard:shard1
[junit4:junit4]   2> 26114 T1622 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4]   2> 26132 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510006 type:delete cxid:0xaa zxid:0xc5 txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4]   2> 26132 T1622 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 26133 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510006 type:create cxid:0xab zxid:0xc6 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 26143 T1622 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 26143 T1622 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 26144 T1622 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:12179/onenodecollectioncore/
[junit4:junit4]   2> 26144 T1622 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 26145 T1622 oasc.SyncStrategy.syncToMe http://127.0.0.1:12179/onenodecollectioncore/ has no replicas
[junit4:junit4]   2> 26145 T1622 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:12179/onenodecollectioncore/
[junit4:junit4]   2> 26145 T1622 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4]   2> 26150 T1591 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d68ae88510006 type:create cxid:0xb5 zxid:0xca txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2>  C341_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:22224_, base_url=http://127.0.0.1:22224}
[junit4:junit4]   2> 26299 T1682 C341 P22224 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:22200/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 26299 T1682 C341 P22224 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:22224 START replicas=[http://127.0.0.1:22200/collection1/] nUpdates=100
[junit4:junit4]   2> 26300 T1682 C341 P22224 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 26300 T1682 C341 P22224 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 26300 T1682 C341 P22224 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 26300 T1682 C341 P22224 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 26300 T1682 C341 P22224 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 26301 T1682 C341 P22224 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:22200/collection1/. core=collection1
[junit4:junit4]   2> 26302 T1682 C341 P22224 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C342 name=collection1 org.apache.solr.core.SolrCore@4dd102ca url=http://127.0.0.1:22200/collection1 node=127.0.0.1:22200_ C342_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:22200_, base_url=http://127.0.0.1:22200, leader=true}
[junit4:junit4]   2> 26309 T1635 C342 P22200 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 26321 T1636 C342 P22200 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 26324 T1636 C342 P22200 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@170153a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dff8a4e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 26324 T1636 C342 P22200 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 26325 T1636 C342 P22200 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@170153a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dff8a4e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@170153a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dff8a4e),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 26325 T1636 C342 P22200 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 26326 T1636 C342 P22200 oass.SolrIndexSearcher.<init> Opening Searcher@27da03d5 realtime
[junit4:junit4]   2> 26326 T1636 C342 P22200 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 26327 T1636 C342 P22200 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 26328 T1682 C341 P22224 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 26329 T1682 C341 P22224 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 26330 T1637 C342 P22200 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 26331 T1637 C342 P22200 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 26331 T1682 C341 P22224 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 26332 T1682 C341 P22224 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 26332 T1682 C341 P22224 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 26334 T1637 C342 P22200 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 26335 T1682 C341 P22224 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 26336 T1682 C341 P22224 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4/index.20130314053527782 forceNew: false
[junit4:junit4]   2> 26336 T1682 C341 P22224 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1563b4d lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e2849af) fullCopy=false
[junit4:junit4]   2> 26340 T1637 C342 P22200 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 26342 T1682 C341 P22224 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 26343 T1682 C341 P22224 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 26343 T1682 C341 P22224 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 26345 T1682 C341 P22224 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@516c2093 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc3c2ba),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@516c2093 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc3c2ba),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 26345 T1682 C341 P22224 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 26345 T1682 C341 P22224 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 26346 T1682 C341 P22224 oass.SolrIndexSearcher.<init> Opening Searcher@742486ae main
[junit4:junit4]   2> 26347 T1681 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@742486ae main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 26347 T1682 C341 P22224 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@1563b4d lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e2849af)
[junit4:junit4]   2> 26348 T1682 C341 P22224 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 26348 T1682 C341 P22224 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 26348 T1682 C341 P22224 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 26348 T1682 C341 P22224 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 26350 T1682 C341 P22224 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 26745 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 26762 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22224_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22224"}
[junit4:junit4]   2> 26765 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 26766 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 26766 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 26766 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 26766 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 26766 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 26771 T1622 oasc.ZkController.register We are http://127.0.0.1:12179/onenodecollectioncore/ and leader is http://127.0.0.1:12179/onenodecollectioncore/
[junit4:junit4]   2> 26771 T1622 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:12179
[junit4:junit4]   2> 26771 T1622 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 26772 T1622 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4]   2> 26774 T1622 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 26775 T1622 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {numShards=1&roles=none&name=onenodecollectioncore&action=CREATE&collection=onenodecollection&dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1363260901452/onenodecollection&wt=javabin&version=2} status=0 QTime=4464 
[junit4:junit4]   2> 26775 T1586 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 26776 T1586 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 27778 T1586 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 28275 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 28276 T1609 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":"none",
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"onenodecollectioncore",
[junit4:junit4]   2> 	  "collection":"onenodecollection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:12179_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:12179"}
[junit4:junit4]   2> 28293 T1608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 28293 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 28293 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 28293 T1644 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 28293 T1628 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 28293 T1615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 28781 T1586 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 28783 T1586 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4]   2> 28783 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C343 name=onenodecollectioncore org.apache.solr.core.SolrCore@c6a716d url=http://127.0.0.1:12179/onenodecollectioncore node=127.0.0.1:12179_ C343_STATE=coll:onenodecollection core:onenodecollectioncore props:{shard=shard1, roles=none, state=active, core=onenodecollectioncore, collection=onenodecollection, node_name=127.0.0.1:12179_, base_url=http://127.0.0.1:12179, leader=true}
[junit4:junit4]   2> 28809 T1623 C343 P12179 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@159ce328 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a70b14c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 28809 T1623 C343 P12179 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 28812 T1623 C343 P12179 /update {wt=javabin&version=2} {add=[1 (1429482693202018304)]} 0 6
[junit4:junit4]   2> 28820 T1623 C343 P12179 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 28823 T1623 C343 P12179 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@159ce328 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a70b14c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@159ce328 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a70b14c),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, segments_2, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4]   2> 28824 T1623 C343 P12179 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, segments_2, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4]   2> 28825 T1623 C343 P12179 oass.SolrIndexSearcher.<init> Opening Searcher@7308bcfa main
[junit4:junit4]   2> 28826 T1623 C343 P12179 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 28827 T1684 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@7308bcfa main{StandardDirectoryReader(segments_2:3 _0(5.0):C1)}
[junit4:junit4]   2> 28827 T1623 C343 P12179 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4]   2> 28831 T1623 C343 P12179 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 28832 T1586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 28847 T1623 C343 P12179 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> ASYNC  NEW_CORE C344 name=collection1 org.apache.solr.core.SolrCore@6bd3162f url=http://127.0.0.1:57771/collection1 node=127.0.0.1:57771_ C344_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:57771_, base_url=http://127.0.0.1:57771, leader=true}
[junit4:junit4]   2> 28858 T1598 C344 P57771 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a7de20b lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e665d45),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 28859 T1598 C344 P57771 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 28864 T1598 C344 P57771 /update {wt=javabin&version=2} {add=[1 (1429482693253398528)]} 0 8
[junit4:junit4]   2>  C339_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:22220_, base_url=http://127.0.0.1:22220}
[junit4:junit4]   2> 28903 T1650 C339 P22220 /update {distrib.from=http://127.0.0.1:12179/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1429482693283807232)]} 0 5
[junit4:junit4]   2> 28904 T1624 C340 P12179 /update {distrib.from=http://127.0.0.1:22200/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1429482693283807232)]} 0 20
[junit4:junit4]   2> 28905 T1640 C342 P22200 /update {wt=javabin&version=2} {add=[1]} 0 25
[junit4:junit4]   2> 28908 T1598 C344 P57771 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 28913 T1598 C344 P57771 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a7de20b lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e665d45),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a7de20b lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e665d45),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, segments_2, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4]   2> 28914 T1598 C344 P57771 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, segments_2, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4]   2> 28916 T1598 C344 P57771 oass.SolrIndexSearcher.<init> Opening Searcher@3d228da4 main
[junit4:junit4]   2> 28916 T1598 C344 P57771 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 28917 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d228da4 main{StandardDirectoryReader(segments_2:3 _0(5.0):C1)}
[junit4:junit4]   2> 28918 T1598 C344 P57771 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4]   2> 28921 T1640 C342 P22200 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 28922 T1640 C342 P22200 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@170153a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dff8a4e),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@170153a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dff8a4e),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 28922 T1640 C342 P22200 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 28923 T1640 C342 P22200 oass.SolrIndexSearcher.<init> Opening Searcher@bc24367 main
[junit4:junit4]   2> 28923 T1640 C342 P22200 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 28924 T1646 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@bc24367 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 28925 T1640 C342 P22200 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:12179/collection1/, StdNode: http://127.0.0.1:22220/collection1/, StdNode: http://127.0.0.1:22224/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 28928 T1651 C339 P22220 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 28928 T1624 C340 P12179 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 28933 T1651 C339 P22220 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cfd75e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d024f4f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cfd75e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d024f4f),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3, _0_dv.cfe]
[junit4:junit4]   2> 28933 T1624 C340 P12179 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12a31785 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a7eda65),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12a31785 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a7eda65),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3, _0_dv.cfe]
[junit4:junit4]   2> 28934 T1651 C339 P22220 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3, _0_dv.cfe]
[junit4:junit4]   2> 28934 T1624 C340 P12179 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3, _0_dv.cfe]
[junit4:junit4]   2>  C341_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:22224_, base_url=http://127.0.0.1:22224}
[junit4:junit4]   2> 28934 T1667 C341 P22224 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 28935 T1667 C341 P22224 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@516c2093 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc3c2ba),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@516c2093 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bc3c2ba),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 28936 T1667 C341 P22224 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 28936 T1651 C339 P22220 oass.SolrIndexSearcher.<init> Opening Searcher@1d66ec1b main
[junit4:junit4]   2> 28936 T1624 C340 P12179 oass.SolrIndexSearcher.<init> Opening Searcher@6de04bb1 main
[junit4:junit4]   2> 28936 T1667 C341 P22224 oass.SolrIndexSearcher.<init> Opening Searcher@2487f8da main
[junit4:junit4]   2> 28937 T1651 C339 P22220 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 28937 T1624 C340 P12179 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 28938 T1630 oasc.JmxMonitoredMap.put WARNING Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4]   2> 	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4]   2> 	at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4]   2> 	at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4]   2> 	at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:239)
[junit4:junit4]   2> 	at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1763)
[junit4:junit4]   2> 	at org.apache.solr.core.SolrCore.access$000(SolrCore.java:129)
[junit4:junit4]   2> 	at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1640)
[junit4:junit4]   2> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 28938 T1662 oasc.JmxMonitoredMap.put WARNING Failed to register info bean: fieldValueCache javax.management.InstanceAlreadyExistsException: solr/collection1:type=fieldValueCache,id=org.apache.solr.search.FastLRUCache
[junit4:junit4]   2> 	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4]   2> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4]   2> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4]   2> 	at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4]   2> 	at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4]   2> 	at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:243)
[junit4:junit4]   2> 	at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1763)
[junit4:junit4]   2> 	at org.apache.solr.core.SolrCore.access$000(SolrCore.java:129)
[junit4:junit4]   2> 	at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1640)
[junit4:junit4]   2> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 28939 T1630 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6de04bb1 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4]   2> 28939 T1662 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d66ec1b main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4]   2> 28937 T1667 C341 P22224 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 28938 T1681 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2487f8da main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 28940 T1651 C339 P22220 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4]   2> 28940 T1624 C340 P12179 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4]   2> 28940 T1667 C341 P22224 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 28941 T1640 C342 P22200 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4]   2> 28941 T1586 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 28943 T1586 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 28944 T1586 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> 28946 T1598 C344 P57771 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 28947 T1586 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44540/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@967a5e
[junit4:junit4]   2> 28949 T1586 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 28949 T1692 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44540. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 28949 T1692 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44540, initiating session
[junit4:junit4]   2> 28950 T1588 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22244
[junit4:junit4]   2> 28950 T1588 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22244
[junit4:junit4]   2> 28955 T1590 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d68ae8851000d with negotiated timeout 10000 for client /140.211.11.196:22244
[junit4:junit4]   2> 28956 T1692 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44540, sessionid = 0x13d68ae8851000d, negotiated timeout = 10000
[junit4:junit4]   2> 28956 T1693 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@967a5e name:ZooKeeperConnection Watcher:127.0.0.1:44540/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 28956 T1586 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 28958 T1586 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 28961 T1591 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d68ae8851000d
[junit4:junit4]   2> 28962 T

[...truncated too long message...]

ol.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 141197 T1586 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 141554 T1589 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4]   2> 157085 T1677 oasc.SolrException.log SEVERE Sync request error: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:22200/collection1
[junit4:junit4]   2> 157086 T1677 oasc.SyncStrategy.syncToMe http://127.0.0.1:22224/collection1/: Sync failed - asking replica (http://127.0.0.1:22200/collection1/) to recover.
[junit4:junit4]   2> 157086 T1677 oasc.SyncStrategy.syncToMe We have been closed, don't request that a replica recover
[junit4:junit4]   2> 157086 T1677 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:22224/collection1/
[junit4:junit4]   2> 157086 T1677 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@319e02e7
[junit4:junit4]   2> 157092 T1677 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=35,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 157093 T1677 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 157093 T1677 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 157094 T1677 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 157096 T1677 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 157096 T1677 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4/index
[junit4:junit4]   2> 157097 T1677 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4/index
[junit4:junit4]   2> 157097 T1677 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4
[junit4:junit4]   2> 157097 T1677 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4
[junit4:junit4]   2> 157097 T1677 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4/index.20130314053527782
[junit4:junit4]   2> 157097 T1677 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1363260901452/jetty4/index.20130314053527782
[junit4:junit4]   2> 157098 T1677 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 157098 T1677 oasc.LeaderElector$1.process WARNING  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election/89342666437885964-127.0.0.1:22224__collection1-n_0000000001
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:153)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:150)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:150)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:243)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> 157099 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 157099 T1677 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 157099 T1677 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 157099 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 157100 T1677 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 157100 T1677 oasc.LeaderElector$1.process WARNING  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:236)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:233)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:233)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> 157100 T1677 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=el, timezone=America/Swift_Current
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_17 (64-bit)/cpus=16,threads=2,free=63770336,total=276824064
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestFoldingMultitermQuery, ChaosMonkeySafeLeaderTest, TestReload, PingRequestHandlerTest, TestLazyCores, DebugComponentTest, FileUtilsTest, SampleTest, MultiTermTest, SignatureUpdateProcessorFactoryTest, SolrRequestParserTest, TestWordDelimiterFilterFactory, TermVectorComponentDistributedTest, TestLuceneMatchVersion, TestAnalyzedSuggestions, TestSolrXMLSerializer, StatsComponentTest, TestMergePolicyConfig, TestSolr4Spatial, BasicZkTest, TimeZoneUtilsTest, SpellCheckComponentTest, RequiredFieldsTest, URLClassifyProcessorTest, JSONWriterTest, TestSuggestSpellingConverter, TestRTGBase, ZkSolrClientTest, TestJmxIntegration, SuggesterWFSTTest, TestQuerySenderNoQuery, XsltUpdateRequestHandlerTest, TestDFRSimilarityFactory, DocumentAnalysisRequestHandlerTest, NumericFieldsTest, CurrencyFieldXmlFileTest, TestRangeQuery, TestQueryTypes, DateMathParserTest, TestDynamicFieldResource, SolrCmdDistributorTest, TestNumberUtils, TestHashPartitioner, NotRequiredUniqueKeyTest, TestRecovery, OpenCloseCoreStressTest, OpenExchangeRatesOrgProviderTest, TestCollationField, SolrPluginUtilsTest, SimplePostToolTest, TestZkChroot, NoCacheHeaderTest, BasicDistributedZk2Test]
[junit4:junit4] Completed on J1 in 157.24s, 1 test, 1 error <<< FAILURES!

[...truncated 460 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:877: There were test failures: 271 suites, 1142 tests, 1 error, 27 ignored (8 assumptions)

Total time: 53 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message