lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 207 - Still Failing
Date Thu, 14 Mar 2013 00:07:30 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/207/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.OpenCloseCoreStressTest

Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=949, name=qtp1882088204-949, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest], registration stack trace below.

Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=949, name=qtp1882088204-949, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest], registration stack trace below.
	at __randomizedtesting.SeedInfo.seed([2BE095131715C45A]:0)
	at java.lang.Thread.getStackTrace(Thread.java:1495)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:520)
	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:986)
	at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:929)
	at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:909)
	at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:904)
	at org.apache.solr.core.MockFSDirectoryFactory.create(MockFSDirectoryFactory.java:34)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:272)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:224)
	at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:124)
	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1405)
	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1541)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:795)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:619)
	at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:927)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:957)
	at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1158)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:226)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:275)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1074)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:365)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
	at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:948)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
	at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
	at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
	at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/OpenCloseCoreStressTest_/00016_core/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@727cc402)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	... 1 more


FAILED:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:38767/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:16138/collection1], CloudJettyRunner [url=http://127.0.0.1:19858/collection1]]

Stack Trace:
java.lang.AssertionError: Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:38767/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:16138/collection1], CloudJettyRunner [url=http://127.0.0.1:19858/collection1]]
	at __randomizedtesting.SeedInfo.seed([2BE095131715C45A:AA061B0B604AA466]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:213)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:679)




Build Log:
[...truncated 8513 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4]   2> 0 T617 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 8 T617 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1363218059210
[junit4:junit4]   2> 10 T617 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 11 T618 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 15 T618 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 15 T618 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 15 T618 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 16 T618 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 17 T618 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1363218059209/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 112 T617 oasc.ZkTestServer.run start zk server on port:35115
[junit4:junit4]   2> 113 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@54dc6a8
[junit4:junit4]   2> 114 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 115 T623 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 115 T623 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 116 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57981
[junit4:junit4]   2> 117 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57981
[junit4:junit4]   2> 117 T621 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 147 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20000 with negotiated timeout 10000 for client /140.211.11.196:57981
[junit4:junit4]   2> 147 T623 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd20000, negotiated timeout = 10000
[junit4:junit4]   2> 147 T624 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54dc6a8 name:ZooKeeperConnection Watcher:127.0.0.1:35115 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 148 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 148 T617 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 153 T622 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6620cfd20000
[junit4:junit4]   2> 154 T624 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 154 T617 oaz.ZooKeeper.close Session: 0x13d6620cfd20000 closed
[junit4:junit4]   2> 156 T619 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57981 which had sessionid 0x13d6620cfd20000
[junit4:junit4]   2> 156 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@12116c50
[junit4:junit4]   2> 157 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 158 T625 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 158 T625 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 159 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12321
[junit4:junit4]   2> 159 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12321
[junit4:junit4]   2> 160 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20001 with negotiated timeout 10000 for client /140.211.11.196:12321
[junit4:junit4]   2> 161 T625 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd20001, negotiated timeout = 10000
[junit4:junit4]   2> 161 T626 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12116c50 name:ZooKeeperConnection Watcher:127.0.0.1:35115/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 162 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 162 T617 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 168 T617 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 173 T617 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 177 T617 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 184 T617 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 185 T617 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 200 T617 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 202 T617 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 210 T617 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 211 T617 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 218 T617 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 219 T617 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 225 T617 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 226 T617 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 232 T617 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 233 T617 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 240 T617 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 241 T617 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 246 T617 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 247 T617 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 253 T617 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 254 T617 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 261 T622 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6620cfd20001
[junit4:junit4]   2> 262 T626 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 262 T619 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:12321 which had sessionid 0x13d6620cfd20001
[junit4:junit4]   2> 262 T617 oaz.ZooKeeper.close Session: 0x13d6620cfd20001 closed
[junit4:junit4]   2> 503 T617 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 510 T617 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:28426
[junit4:junit4]   2> 511 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 512 T617 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 512 T617 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363218059464
[junit4:junit4]   2> 513 T617 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363218059464/solr.xml
[junit4:junit4]   2> 513 T617 oasc.CoreContainer.<init> New CoreContainer 591092348
[junit4:junit4]   2> 514 T617 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363218059464/'
[junit4:junit4]   2> 515 T617 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363218059464/'
[junit4:junit4]   2> 600 T617 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 600 T617 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 601 T617 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 602 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 602 T617 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 603 T617 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 604 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 604 T617 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 605 T617 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 606 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 649 T617 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 693 T617 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35115/solr
[junit4:junit4]   2> 694 T617 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 695 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5aaa980
[junit4:junit4]   2> 697 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 698 T636 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 701 T636 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 702 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:42560
[junit4:junit4]   2> 702 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:42560
[junit4:junit4]   2> 705 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20002 with negotiated timeout 20000 for client /140.211.11.196:42560
[junit4:junit4]   2> 705 T636 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd20002, negotiated timeout = 20000
[junit4:junit4]   2> 705 T637 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5aaa980 name:ZooKeeperConnection Watcher:127.0.0.1:35115 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 706 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 709 T622 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6620cfd20002
[junit4:junit4]   2> 711 T637 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 711 T619 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:42560 which had sessionid 0x13d6620cfd20002
[junit4:junit4]   2> 711 T617 oaz.ZooKeeper.close Session: 0x13d6620cfd20002 closed
[junit4:junit4]   2> 713 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 754 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3175252f
[junit4:junit4]   2> 756 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 756 T638 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 757 T638 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 757 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:17261
[junit4:junit4]   2> 758 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:17261
[junit4:junit4]   2> 760 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20003 with negotiated timeout 20000 for client /140.211.11.196:17261
[junit4:junit4]   2> 760 T638 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd20003, negotiated timeout = 20000
[junit4:junit4]   2> 762 T639 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3175252f name:ZooKeeperConnection Watcher:127.0.0.1:35115/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 762 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 767 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 771 T617 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 777 T617 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:28426_
[junit4:junit4]   2> 779 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:28426_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:28426_
[junit4:junit4]   2> 781 T617 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:28426_
[junit4:junit4]   2> 789 T617 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 804 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 807 T617 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 814 T617 oasc.Overseer.start Overseer (id=89339858728910851-127.0.0.1:28426_-n_0000000000) starting
[junit4:junit4]   2> 816 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 818 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 822 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 824 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 832 T641 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 834 T617 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 841 T617 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 847 T617 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 857 T640 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 873 T642 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363218059464/collection1
[junit4:junit4]   2> 874 T642 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 876 T642 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 876 T642 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 882 T642 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363218059464/collection1/'
[junit4:junit4]   2> 885 T642 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1363218059464/collection1/lib/README' to classloader
[junit4:junit4]   2> 885 T642 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1363218059464/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1039 T642 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 1183 T642 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1185 T642 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1200 T642 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2200 T642 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2216 T642 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2221 T642 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2248 T642 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2256 T642 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2264 T642 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2267 T642 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2267 T642 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 2268 T642 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2270 T642 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2271 T642 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 2271 T642 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2272 T642 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363218059464/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/control/data/
[junit4:junit4]   2> 2272 T642 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79024994
[junit4:junit4]   2> 2273 T642 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2274 T642 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/control/data forceNew: false
[junit4:junit4]   2> 2274 T642 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363218059209/control/data/index/
[junit4:junit4]   2> 2275 T642 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363218059209/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2275 T642 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/control/data/index forceNew: false
[junit4:junit4]   2> 2281 T642 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3aea91be lockFactory=org.apache.lucene.store.NativeFSLockFactory@586c2d62),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2281 T642 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 2285 T642 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2285 T642 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2286 T642 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2287 T642 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2288 T642 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2288 T642 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2289 T642 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2290 T642 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2290 T642 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2303 T642 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2313 T642 oass.SolrIndexSearcher.<init> Opening Searcher@57410e49 main
[junit4:junit4]   2> 2314 T642 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/control/data/tlog
[junit4:junit4]   2> 2315 T642 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2315 T642 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2321 T643 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@57410e49 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2325 T642 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 2325 T642 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2326 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:create cxid:0x55 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2410 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2411 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:28426_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:28426"}
[junit4:junit4]   2> 2412 T640 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2412 T640 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2413 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:create cxid:0x5d zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2435 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3409 T642 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 3409 T642 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:28426 collection:control_collection shard:shard1
[junit4:junit4]   2> 3411 T642 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 3455 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:delete cxid:0x77 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 3457 T642 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 3458 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:create cxid:0x78 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3469 T642 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 3469 T642 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 3470 T642 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:28426/collection1/
[junit4:junit4]   2> 3470 T642 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 3470 T642 oasc.SyncStrategy.syncToMe http://127.0.0.1:28426/collection1/ has no replicas
[junit4:junit4]   2> 3470 T642 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:28426/collection1/
[junit4:junit4]   2> 3471 T642 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 3477 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20003 type:create cxid:0x82 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3945 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3966 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4003 T642 oasc.ZkController.register We are http://127.0.0.1:28426/collection1/ and leader is http://127.0.0.1:28426/collection1/
[junit4:junit4]   2> 4003 T642 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:28426
[junit4:junit4]   2> 4003 T642 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 4004 T642 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 4004 T642 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4007 T642 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4008 T617 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 4009 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4010 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4031 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 4033 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@18369f6d
[junit4:junit4]   2> 4034 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4035 T645 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 4035 T645 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 4036 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:60081
[junit4:junit4]   2> 4036 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:60081
[junit4:junit4]   2> 4038 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20004 with negotiated timeout 10000 for client /140.211.11.196:60081
[junit4:junit4]   2> 4038 T645 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd20004, negotiated timeout = 10000
[junit4:junit4]   2> 4038 T646 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18369f6d name:ZooKeeperConnection Watcher:127.0.0.1:35115/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4039 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4041 T617 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4045 T617 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4283 T617 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4286 T617 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38368
[junit4:junit4]   2> 4287 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4288 T617 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4288 T617 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363218063245
[junit4:junit4]   2> 4289 T617 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363218063245/solr.xml
[junit4:junit4]   2> 4289 T617 oasc.CoreContainer.<init> New CoreContainer 494633917
[junit4:junit4]   2> 4290 T617 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363218063245/'
[junit4:junit4]   2> 4291 T617 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363218063245/'
[junit4:junit4]   2> 4352 T617 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 4353 T617 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4353 T617 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4354 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4354 T617 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4355 T617 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4355 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4356 T617 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4357 T617 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4358 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4385 T617 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4413 T617 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35115/solr
[junit4:junit4]   2> 4414 T617 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4414 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@55bf573b
[junit4:junit4]   2> 4415 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4416 T656 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 4417 T656 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 4417 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:51541
[junit4:junit4]   2> 4417 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:51541
[junit4:junit4]   2> 4420 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20005 with negotiated timeout 20000 for client /140.211.11.196:51541
[junit4:junit4]   2> 4420 T656 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd20005, negotiated timeout = 20000
[junit4:junit4]   2> 4420 T657 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55bf573b name:ZooKeeperConnection Watcher:127.0.0.1:35115 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4421 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4423 T622 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6620cfd20005
[junit4:junit4]   2> 4432 T657 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4432 T619 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:51541 which had sessionid 0x13d6620cfd20005
[junit4:junit4]   2> 4432 T617 oaz.ZooKeeper.close Session: 0x13d6620cfd20005 closed
[junit4:junit4]   2> 4434 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 4446 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3e75a7f3
[junit4:junit4]   2> 4448 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4448 T658 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 4449 T658 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 4449 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10294
[junit4:junit4]   2> 4450 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10294
[junit4:junit4]   2> 4452 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20006 with negotiated timeout 20000 for client /140.211.11.196:10294
[junit4:junit4]   2> 4452 T658 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd20006, negotiated timeout = 20000
[junit4:junit4]   2> 4452 T659 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e75a7f3 name:ZooKeeperConnection Watcher:127.0.0.1:35115/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4452 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4454 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20006 type:create cxid:0x1 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4456 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20006 type:create cxid:0x2 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4459 T617 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5465 T617 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38368_
[junit4:junit4]   2> 5466 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20006 type:delete cxid:0xb zxid:0x4c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38368_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38368_
[junit4:junit4]   2> 5494 T617 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38368_
[junit4:junit4]   2> 5496 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5497 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:28426_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:28426"}
[junit4:junit4]   2> 5499 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 5509 T659 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5509 T646 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5524 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5524 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5528 T660 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363218063245/collection1
[junit4:junit4]   2> 5528 T660 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5529 T660 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5529 T660 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5530 T639 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5530 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5531 T660 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363218063245/collection1/'
[junit4:junit4]   2> 5531 T660 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1363218063245/collection1/lib/README' to classloader
[junit4:junit4]   2> 5532 T660 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1363218063245/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5575 T660 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 5641 T660 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5643 T660 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5654 T660 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6442 T660 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6459 T660 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6465 T660 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6493 T660 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6500 T660 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6508 T660 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6510 T660 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 6511 T660 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 6511 T660 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6513 T660 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 6514 T660 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 6514 T660 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 6514 T660 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363218063245/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty1/
[junit4:junit4]   2> 6515 T660 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79024994
[junit4:junit4]   2> 6516 T660 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 6516 T660 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty1 forceNew: false
[junit4:junit4]   2> 6517 T660 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty1/index/
[junit4:junit4]   2> 6517 T660 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6518 T660 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty1/index forceNew: false
[junit4:junit4]   2> 6521 T660 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2cd39e6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4873daf9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6522 T660 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 6526 T660 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6526 T660 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6527 T660 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6528 T660 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6529 T660 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6530 T660 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6530 T660 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6531 T660 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6532 T660 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6542 T660 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6550 T660 oass.SolrIndexSearcher.<init> Opening Searcher@1d727f16 main
[junit4:junit4]   2> 6550 T660 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty1/tlog
[junit4:junit4]   2> 6552 T660 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6552 T660 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6559 T661 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d727f16 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6563 T660 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 6563 T660 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7033 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7034 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38368_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38368"}
[junit4:junit4]   2> 7034 T640 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 7034 T640 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 7065 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7065 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7065 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7594 T660 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7594 T660 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38368 collection:collection1 shard:shard1
[junit4:junit4]   2> 7595 T660 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7626 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20006 type:delete cxid:0x48 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7632 T660 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7633 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20006 type:create cxid:0x49 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7637 T660 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7637 T660 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7637 T660 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38368/collection1/
[junit4:junit4]   2> 7638 T660 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7638 T660 oasc.SyncStrategy.syncToMe http://127.0.0.1:38368/collection1/ has no replicas
[junit4:junit4]   2> 7639 T660 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38368/collection1/
[junit4:junit4]   2> 7639 T660 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7647 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20006 type:create cxid:0x53 zxid:0x61 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8575 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8632 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8632 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8632 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8678 T660 oasc.ZkController.register We are http://127.0.0.1:38368/collection1/ and leader is http://127.0.0.1:38368/collection1/
[junit4:junit4]   2> 8678 T660 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38368
[junit4:junit4]   2> 8678 T660 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8678 T660 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8679 T660 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8681 T660 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8682 T617 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8683 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8684 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8932 T617 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8936 T617 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38767
[junit4:junit4]   2> 8937 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8938 T617 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8939 T617 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363218067905
[junit4:junit4]   2> 8939 T617 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363218067905/solr.xml
[junit4:junit4]   2> 8940 T617 oasc.CoreContainer.<init> New CoreContainer 1732922169
[junit4:junit4]   2> 8940 T617 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363218067905/'
[junit4:junit4]   2> 8941 T617 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363218067905/'
[junit4:junit4]   2> 9001 T617 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 9002 T617 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9003 T617 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 9003 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9004 T617 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9004 T617 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9005 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9005 T617 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9006 T617 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9007 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 9034 T617 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9062 T617 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35115/solr
[junit4:junit4]   2> 9063 T617 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 9063 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3663c64f
[junit4:junit4]   2> 9065 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9065 T672 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 9067 T672 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 9067 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:23847
[junit4:junit4]   2> 9067 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:23847
[junit4:junit4]   2> 9070 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20007 with negotiated timeout 20000 for client /140.211.11.196:23847
[junit4:junit4]   2> 9070 T672 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd20007, negotiated timeout = 20000
[junit4:junit4]   2> 9070 T673 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3663c64f name:ZooKeeperConnection Watcher:127.0.0.1:35115 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9070 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9072 T622 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6620cfd20007
[junit4:junit4]   2> 9085 T673 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 9085 T619 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:23847 which had sessionid 0x13d6620cfd20007
[junit4:junit4]   2> 9085 T617 oaz.ZooKeeper.close Session: 0x13d6620cfd20007 closed
[junit4:junit4]   2> 9086 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 9094 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@45f0068e
[junit4:junit4]   2> 9095 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9095 T674 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 9096 T674 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 9096 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11432
[junit4:junit4]   2> 9096 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11432
[junit4:junit4]   2> 9098 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20008 with negotiated timeout 20000 for client /140.211.11.196:11432
[junit4:junit4]   2> 9098 T674 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd20008, negotiated timeout = 20000
[junit4:junit4]   2> 9098 T675 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45f0068e name:ZooKeeperConnection Watcher:127.0.0.1:35115/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9098 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9100 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20008 type:create cxid:0x1 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9102 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20008 type:create cxid:0x2 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9105 T617 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10111 T617 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38767_
[junit4:junit4]   2> 10112 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd20008 type:delete cxid:0xb zxid:0x70 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38767_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38767_
[junit4:junit4]   2> 10140 T617 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38767_
[junit4:junit4]   2> 10144 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10144 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10145 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10145 T659 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 10146 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10146 T675 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 10146 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38368_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38368"}
[junit4:junit4]   2> 10153 T646 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 10162 T676 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363218067905/collection1
[junit4:junit4]   2> 10163 T676 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10163 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10163 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10163 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10164 T676 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10165 T676 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10168 T676 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363218067905/collection1/'
[junit4:junit4]   2> 10170 T676 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1363218067905/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10171 T676 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1363218067905/collection1/lib/README' to classloader
[junit4:junit4]   2> 10171 T639 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 10171 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10231 T676 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 10305 T676 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10306 T676 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10316 T676 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11330 T676 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11352 T676 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11358 T676 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11415 T676 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11425 T676 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11433 T676 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11436 T676 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 11437 T676 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 11437 T676 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11439 T676 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 11439 T676 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 11440 T676 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 11440 T676 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363218067905/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty2/
[junit4:junit4]   2> 11441 T676 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79024994
[junit4:junit4]   2> 11441 T676 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 11443 T676 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty2 forceNew: false
[junit4:junit4]   2> 11443 T676 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty2/index/
[junit4:junit4]   2> 11443 T676 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11444 T676 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty2/index forceNew: false
[junit4:junit4]   2> 11449 T676 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7f6578c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@307cffec),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11449 T676 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11453 T676 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11454 T676 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11455 T676 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11456 T676 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11457 T676 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11457 T676 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11458 T676 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11459 T676 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11460 T676 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11477 T676 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11490 T676 oass.SolrIndexSearcher.<init> Opening Searcher@6522008b main
[junit4:junit4]   2> 11490 T676 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty2/tlog
[junit4:junit4]   2> 11491 T676 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11492 T676 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11522 T677 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6522008b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11527 T676 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11527 T676 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11671 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11673 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38767_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38767"}
[junit4:junit4]   2> 11673 T640 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 11673 T640 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11683 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11683 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11683 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11684 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12531 T676 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12531 T676 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38767 collection:collection1 shard:shard1
[junit4:junit4]   2> 12557 T676 oasc.ZkController.register We are http://127.0.0.1:38767/collection1/ and leader is http://127.0.0.1:38368/collection1/
[junit4:junit4]   2> 12558 T676 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38767
[junit4:junit4]   2> 12558 T676 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 12558 T676 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C21 name=collection1 org.apache.solr.core.SolrCore@8f7ad17 url=http://127.0.0.1:38767/collection1 node=127.0.0.1:38767_ C21_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:38767_, base_url=http://127.0.0.1:38767}
[junit4:junit4]   2> 12559 T678 C21 P38767 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 12561 T676 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12561 T678 C21 P38767 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 12561 T678 C21 P38767 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 12562 T678 C21 P38767 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12564 T617 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12565 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12566 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12566 T678 C21 P38767 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12891 T617 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12895 T617 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:29531
[junit4:junit4]   2> 12896 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12897 T617 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12898 T617 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363218071799
[junit4:junit4]   2> 12898 T617 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363218071799/solr.xml
[junit4:junit4]   2> 12899 T617 oasc.CoreContainer.<init> New CoreContainer 343386315
[junit4:junit4]   2> 12900 T617 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363218071799/'
[junit4:junit4]   2> 12901 T617 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363218071799/'
[junit4:junit4]   2> 12977 T617 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 12978 T617 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12978 T617 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12979 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12980 T617 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12980 T617 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12981 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12982 T617 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12982 T617 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12983 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 13016 T617 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13043 T617 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35115/solr
[junit4:junit4]   2> 13044 T617 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 13044 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2d471baf
[junit4:junit4]   2> 13046 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13046 T689 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 13047 T689 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 13048 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58024
[junit4:junit4]   2> 13048 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58024
[junit4:junit4]   2> 13051 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20009 with negotiated timeout 20000 for client /140.211.11.196:58024
[junit4:junit4]   2> 13051 T689 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd20009, negotiated timeout = 20000
[junit4:junit4]   2> 13052 T690 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d471baf name:ZooKeeperConnection Watcher:127.0.0.1:35115 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13052 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13054 T622 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6620cfd20009
[junit4:junit4]   2> 13057 T619 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:58024 which had sessionid 0x13d6620cfd20009
[junit4:junit4]   2> 13057 T617 oaz.ZooKeeper.close Session: 0x13d6620cfd20009 closed
[junit4:junit4]   2> 13057 T690 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 13058 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 13078 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6d0840da
[junit4:junit4]   2> 13080 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13081 T691 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 13082 T691 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 13082 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:28892
[junit4:junit4]   2> 13083 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:28892
[junit4:junit4]   2> 13085 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd2000a with negotiated timeout 20000 for client /140.211.11.196:28892
[junit4:junit4]   2> 13085 T691 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd2000a, negotiated timeout = 20000
[junit4:junit4]   2> 13085 T692 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d0840da name:ZooKeeperConnection Watcher:127.0.0.1:35115/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13086 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13090 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd2000a type:create cxid:0x1 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13092 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd2000a type:create cxid:0x2 zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13098 T617 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13196 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13198 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38767_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38767"}
[junit4:junit4]   2> 13212 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13212 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13212 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13212 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13213 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13602 T649 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=3&state=recovering&nodeName=127.0.0.1:38767_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001 
[junit4:junit4]   2> 14104 T617 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:29531_
[junit4:junit4]   2> 14105 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd2000a type:delete cxid:0xd zxid:0x87 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:29531_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:29531_
[junit4:junit4]   2> 14138 T617 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:29531_
[junit4:junit4]   2> 14142 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14142 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14143 T659 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14143 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14143 T675 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14143 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14144 T692 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14144 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14145 T639 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14146 T646 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14161 T693 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363218071799/collection1
[junit4:junit4]   2> 14162 T693 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 14163 T693 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 14163 T693 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 14166 T693 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363218071799/collection1/'
[junit4:junit4]   2> 14168 T693 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1363218071799/collection1/lib/README' to classloader
[junit4:junit4]   2> 14169 T693 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1363218071799/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 14237 T693 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 14308 T693 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 14310 T693 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 14319 T693 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 15166 T693 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 15186 T693 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 15193 T693 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 15220 T693 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15228 T693 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15236 T693 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15238 T693 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 15238 T693 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 15239 T693 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15241 T693 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 15241 T693 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 15241 T693 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 15242 T693 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363218071799/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty3/
[junit4:junit4]   2> 15242 T693 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79024994
[junit4:junit4]   2> 15243 T693 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 15244 T693 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty3 forceNew: false
[junit4:junit4]   2> 15244 T693 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty3/index/
[junit4:junit4]   2> 15245 T693 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 15245 T693 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty3/index forceNew: false
[junit4:junit4]   2> 15250 T693 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5463f35f lockFactory=org.apache.lucene.store.NativeFSLockFactory@3bf5ed2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15250 T693 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15255 T693 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 15255 T693 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 15256 T693 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 15257 T693 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 15258 T693 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 15259 T693 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 15259 T693 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 15260 T693 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 15261 T693 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 15273 T693 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 15282 T693 oass.SolrIndexSearcher.<init> Opening Searcher@27e67ef3 main
[junit4:junit4]   2> 15283 T693 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty3/tlog
[junit4:junit4]   2> 15284 T693 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 15285 T693 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 15292 T694 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@27e67ef3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 15295 T693 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 15295 T693 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2>  C21_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:38767_, base_url=http://127.0.0.1:38767}
[junit4:junit4]   2> 15604 T678 C21 P38767 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:38368/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15604 T678 C21 P38767 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38767 START replicas=[http://127.0.0.1:38368/collection1/] nUpdates=100
[junit4:junit4]   2> 15605 T678 C21 P38767 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15606 T678 C21 P38767 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15607 T678 C21 P38767 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15607 T678 C21 P38767 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15608 T678 C21 P38767 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15608 T678 C21 P38767 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:38368/collection1/. core=collection1
[junit4:junit4]   2> 15608 T678 C21 P38767 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C22 name=collection1 org.apache.solr.core.SolrCore@2b663f09 url=http://127.0.0.1:38368/collection1 node=127.0.0.1:38368_ C22_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:38368_, base_url=http://127.0.0.1:38368, leader=true}
[junit4:junit4]   2> 15617 T650 C22 P38368 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15641 T651 C22 P38368 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 15646 T651 C22 P38368 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2cd39e6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4873daf9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15647 T651 C22 P38368 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15648 T651 C22 P38368 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2cd39e6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4873daf9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2cd39e6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4873daf9),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15648 T651 C22 P38368 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15649 T651 C22 P38368 oass.SolrIndexSearcher.<init> Opening Searcher@4062ce9b realtime
[junit4:junit4]   2> 15650 T651 C22 P38368 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15650 T651 C22 P38368 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4]   2> 15652 T678 C21 P38767 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15652 T678 C21 P38767 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15696 T652 C22 P38368 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15697 T652 C22 P38368 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15699 T678 C21 P38767 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15699 T678 C21 P38767 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15699 T678 C21 P38767 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15703 T652 C22 P38368 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15704 T678 C21 P38767 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15706 T678 C21 P38767 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty2/index.20130313234114905 forceNew: false
[junit4:junit4]   2> 15706 T678 C21 P38767 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@69a90966 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d6ced76) fullCopy=false
[junit4:junit4]   2> 15710 T652 C22 P38368 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 15711 T678 C21 P38767 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15713 T678 C21 P38767 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15714 T678 C21 P38767 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 15728 T678 C21 P38767 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7f6578c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@307cffec),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7f6578c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@307cffec),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15729 T678 C21 P38767 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15729 T678 C21 P38767 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 15730 T678 C21 P38767 oass.SolrIndexSearcher.<init> Opening Searcher@30d4c046 main
[junit4:junit4]   2> 15732 T677 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@30d4c046 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 15733 T678 C21 P38767 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@69a90966 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d6ced76)
[junit4:junit4]   2> 15733 T678 C21 P38767 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 15733 T678 C21 P38767 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 15733 T678 C21 P38767 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15734 T678 C21 P38767 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15878 T678 C21 P38767 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 16228 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16229 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:29531_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:29531"}
[junit4:junit4]   2> 16230 T640 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 16230 T640 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 16244 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38767_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38767"}
[junit4:junit4]   2> 16257 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16257 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16258 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16257 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16258 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16299 T693 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 16299 T693 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:29531 collection:collection1 shard:shard1
[junit4:junit4]   2> 16308 T693 oasc.ZkController.register We are http://127.0.0.1:29531/collection1/ and leader is http://127.0.0.1:38368/collection1/
[junit4:junit4]   2> 16308 T693 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:29531
[junit4:junit4]   2> 16309 T693 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 16309 T693 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C23 name=collection1 org.apache.solr.core.SolrCore@2b027c27 url=http://127.0.0.1:29531/collection1 node=127.0.0.1:29531_ C23_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:29531_, base_url=http://127.0.0.1:29531}
[junit4:junit4]   2> 16309 T697 C23 P29531 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 16310 T697 C23 P29531 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 16311 T697 C23 P29531 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 16311 T693 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16311 T697 C23 P29531 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16314 T617 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 16315 T697 C23 P29531 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16315 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 16316 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16684 T617 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 16687 T617 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:16138
[junit4:junit4]   2> 16688 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 16689 T617 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 16690 T617 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363218075537
[junit4:junit4]   2> 16690 T617 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363218075537/solr.xml
[junit4:junit4]   2> 16691 T617 oasc.CoreContainer.<init> New CoreContainer 627422048
[junit4:junit4]   2> 16692 T617 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363218075537/'
[junit4:junit4]   2> 16693 T617 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363218075537/'
[junit4:junit4]   2> 16768 T617 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 16769 T617 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 16769 T617 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 16770 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 16771 T617 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 16771 T617 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 16772 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 16773 T617 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 16773 T617 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 16774 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 16815 T617 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 16850 T617 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35115/solr
[junit4:junit4]   2> 16851 T617 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 16852 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@368ceb0c
[junit4:junit4]   2> 16854 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 16856 T708 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 16857 T708 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 16858 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:32619
[junit4:junit4]   2> 16858 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:32619
[junit4:junit4]   2> 16870 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd2000b with negotiated timeout 20000 for client /140.211.11.196:32619
[junit4:junit4]   2> 16872 T708 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd2000b, negotiated timeout = 20000
[junit4:junit4]   2> 16873 T709 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@368ceb0c name:ZooKeeperConnection Watcher:127.0.0.1:35115 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 16873 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 16876 T622 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6620cfd2000b
[junit4:junit4]   2> 16878 T709 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 16879 T619 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:32619 which had sessionid 0x13d6620cfd2000b
[junit4:junit4]   2> 16878 T617 oaz.ZooKeeper.close Session: 0x13d6620cfd2000b closed
[junit4:junit4]   2> 16881 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 16898 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@106d908c
[junit4:junit4]   2> 16901 T710 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 16902 T710 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 16902 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 16902 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49222
[junit4:junit4]   2> 16903 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49222
[junit4:junit4]   2> 16906 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd2000c with negotiated timeout 20000 for client /140.211.11.196:49222
[junit4:junit4]   2> 16906 T710 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd2000c, negotiated timeout = 20000
[junit4:junit4]   2> 16907 T711 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@106d908c name:ZooKeeperConnection Watcher:127.0.0.1:35115/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 16907 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 16909 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd2000c type:create cxid:0x1 zxid:0x98 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16918 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd2000c type:create cxid:0x2 zxid:0x99 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16945 T617 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 17772 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17773 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:29531_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:29531"}
[junit4:junit4]   2> 17784 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17784 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17784 T711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17784 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17785 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17784 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17951 T617 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:16138_
[junit4:junit4]   2> 17953 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd2000c type:delete cxid:0xd zxid:0x9e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:16138_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:16138_
[junit4:junit4]   2> 17955 T617 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:16138_
[junit4:junit4]   2> 17960 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17960 T711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17961 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17961 T659 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17961 T675 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17961 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17961 T692 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17961 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17962 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17964 T639 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17964 T711 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17964 T646 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17973 T712 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363218075537/collection1
[junit4:junit4]   2> 17973 T712 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 17975 T712 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 17975 T712 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 17978 T712 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363218075537/collection1/'
[junit4:junit4]   2> 17979 T712 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1363218075537/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 17980 T712 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1363218075537/collection1/lib/README' to classloader
[junit4:junit4]   2> 18046 T712 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 18126 T712 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 18128 T712 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 18139 T712 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 18338 T653 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=4&state=recovering&nodeName=127.0.0.1:29531_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002 
[junit4:junit4]   2> 19021 T712 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 19040 T712 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 19046 T712 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 19074 T712 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 19082 T712 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 19089 T712 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 19093 T712 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 19093 T712 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 19094 T712 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 19096 T712 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 19096 T712 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 19096 T712 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 19097 T712 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363218075537/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty4/
[junit4:junit4]   2> 19097 T712 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79024994
[junit4:junit4]   2> 19098 T712 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 19102 T712 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty4 forceNew: false
[junit4:junit4]   2> 19102 T712 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty4/index/
[junit4:junit4]   2> 19102 T712 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 19103 T712 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty4/index forceNew: false
[junit4:junit4]   2> 19107 T712 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5acc4401 lockFactory=org.apache.lucene.store.NativeFSLockFactory@367bbec2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19108 T712 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 19111 T712 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 19112 T712 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 19113 T712 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 19114 T712 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 19115 T712 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 19115 T712 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 19116 T712 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 19117 T712 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 19117 T712 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 19128 T712 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 19138 T712 oass.SolrIndexSearcher.<init> Opening Searcher@5683b55e main
[junit4:junit4]   2> 19138 T712 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty4/tlog
[junit4:junit4]   2> 19140 T712 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 19140 T712 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 19148 T713 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5683b55e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 19152 T712 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 19153 T712 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19301 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19302 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:16138_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:16138"}
[junit4:junit4]   2> 19302 T640 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 19303 T640 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 19340 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19340 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19340 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19340 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19340 T711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19340 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20184 T712 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 20184 T712 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:16138 collection:collection1 shard:shard1
[junit4:junit4]   2> 20194 T712 oasc.ZkController.register We are http://127.0.0.1:16138/collection1/ and leader is http://127.0.0.1:38368/collection1/
[junit4:junit4]   2> 20194 T712 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:16138
[junit4:junit4]   2> 20194 T712 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 20195 T712 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C24 name=collection1 org.apache.solr.core.SolrCore@12065f1a url=http://127.0.0.1:16138/collection1 node=127.0.0.1:16138_ C24_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:16138_, base_url=http://127.0.0.1:16138}
[junit4:junit4]   2> 20195 T714 C24 P16138 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 20197 T714 C24 P16138 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 20197 T714 C24 P16138 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 20197 T712 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20197 T714 C24 P16138 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20199 T617 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 20200 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 20200 T714 C24 P16138 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20201 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2>  C23_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:29531_, base_url=http://127.0.0.1:29531}
[junit4:junit4]   2> 20340 T697 C23 P29531 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:38368/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 20340 T697 C23 P29531 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:29531 START replicas=[http://127.0.0.1:38368/collection1/] nUpdates=100
[junit4:junit4]   2> 20341 T697 C23 P29531 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 20342 T697 C23 P29531 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 20342 T697 C23 P29531 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 20343 T697 C23 P29531 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 20343 T697 C23 P29531 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 20343 T697 C23 P29531 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:38368/collection1/. core=collection1
[junit4:junit4]   2> 20344 T697 C23 P29531 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20345 T650 C22 P38368 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 20366 T655 C22 P38368 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 20368 T655 C22 P38368 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2cd39e6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4873daf9),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2cd39e6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4873daf9),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 20369 T655 C22 P38368 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 20370 T655 C22 P38368 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20370 T655 C22 P38368 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4]   2> 20372 T697 C23 P29531 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20372 T697 C23 P29531 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 20376 T652 C22 P38368 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 20377 T697 C23 P29531 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4]   2> 20377 T697 C23 P29531 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 20378 T697 C23 P29531 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 20381 T652 C22 P38368 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1 
[junit4:junit4]   2> 20382 T697 C23 P29531 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 20384 T697 C23 P29531 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty3/index.20130313234119583 forceNew: false
[junit4:junit4]   2> 20384 T697 C23 P29531 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@64dd7a0e lockFactory=org.apache.lucene.store.NativeFSLockFactory@171f932a) fullCopy=false
[junit4:junit4]   2> 20388 T652 C22 P38368 REQ /replication {file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=1 
[junit4:junit4]   2> 20390 T697 C23 P29531 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 20392 T697 C23 P29531 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 20392 T697 C23 P29531 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 20395 T697 C23 P29531 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5463f35f lockFactory=org.apache.lucene.store.NativeFSLockFactory@3bf5ed2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5463f35f lockFactory=org.apache.lucene.store.NativeFSLockFactory@3bf5ed2),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 20396 T697 C23 P29531 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 20396 T697 C23 P29531 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 20396 T697 C23 P29531 oass.SolrIndexSearcher.<init> Opening Searcher@1541703c main
[junit4:junit4]   2> 20398 T694 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1541703c main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4]   2> 20399 T697 C23 P29531 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@64dd7a0e lockFactory=org.apache.lucene.store.NativeFSLockFactory@171f932a)
[junit4:junit4]   2> 20399 T697 C23 P29531 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 20400 T697 C23 P29531 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 20400 T697 C23 P29531 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 20400 T697 C23 P29531 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20403 T697 C23 P29531 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 20525 T617 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 20528 T617 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:49323
[junit4:junit4]   2> 20529 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 20529 T617 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 20530 T617 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363218079431
[junit4:junit4]   2> 20530 T617 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363218079431/solr.xml
[junit4:junit4]   2> 20531 T617 oasc.CoreContainer.<init> New CoreContainer 960107123
[junit4:junit4]   2> 20532 T617 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363218079431/'
[junit4:junit4]   2> 20532 T617 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363218079431/'
[junit4:junit4]   2> 20602 T617 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 20603 T617 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 20603 T617 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 20604 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 20605 T617 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 20605 T617 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 20606 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 20607 T617 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 20607 T617 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 20608 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 20646 T617 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 20677 T617 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35115/solr
[junit4:junit4]   2> 20678 T617 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 20678 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2b827602
[junit4:junit4]   2> 20680 T727 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 20680 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 20682 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:28640
[junit4:junit4]   2> 20681 T727 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 20683 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:28640
[junit4:junit4]   2> 20685 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd2000d with negotiated timeout 20000 for client /140.211.11.196:28640
[junit4:junit4]   2> 20685 T727 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd2000d, negotiated timeout = 20000
[junit4:junit4]   2> 20686 T728 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b827602 name:ZooKeeperConnection Watcher:127.0.0.1:35115 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 20686 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 20688 T622 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6620cfd2000d
[junit4:junit4]   2> 20690 T728 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 20690 T619 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:28640 which had sessionid 0x13d6620cfd2000d
[junit4:junit4]   2> 20690 T617 oaz.ZooKeeper.close Session: 0x13d6620cfd2000d closed
[junit4:junit4]   2> 20691 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 20713 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4e8659e9
[junit4:junit4]   2> 20715 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 20715 T729 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 20716 T729 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 20716 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:15895
[junit4:junit4]   2> 20717 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:15895
[junit4:junit4]   2> 20719 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd2000e with negotiated timeout 20000 for client /140.211.11.196:15895
[junit4:junit4]   2> 20720 T729 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd2000e, negotiated timeout = 20000
[junit4:junit4]   2> 20720 T730 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e8659e9 name:ZooKeeperConnection Watcher:127.0.0.1:35115/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 20720 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 20723 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd2000e type:create cxid:0x1 zxid:0xac txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 20725 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd2000e type:create cxid:0x2 zxid:0xad txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 20729 T617 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 20853 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20856 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:16138_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:16138"}
[junit4:junit4]   2> 20902 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:29531_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:29531"}
[junit4:junit4]   2> 20913 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20913 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20913 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20913 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20913 T711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20913 T730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20913 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21236 T654 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=5&state=recovering&nodeName=127.0.0.1:16138_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001 
[junit4:junit4]   2> 21737 T617 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49323_
[junit4:junit4]   2> 21738 T622 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d6620cfd2000e type:delete cxid:0xd zxid:0xb5 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:49323_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:49323_
[junit4:junit4]   2> 21757 T617 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49323_
[junit4:junit4]   2> 21809 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21809 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21809 T711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21810 T659 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 21811 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 21811 T692 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 21811 T730 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 21811 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 21812 T675 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 21812 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 21813 T730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 21813 T646 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 21814 T639 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 21813 T711 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 21844 T731 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363218079431/collection1
[junit4:junit4]   2> 21845 T731 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 21846 T731 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 21846 T731 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 21849 T731 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363218079431/collection1/'
[junit4:junit4]   2> 21851 T731 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty5-1363218079431/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 21853 T731 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty5-1363218079431/collection1/lib/README' to classloader
[junit4:junit4]   2> 21982 T731 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   2> 22079 T731 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 22080 T731 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 22093 T731 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 23031 T731 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 23047 T731 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 23052 T731 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 23077 T731 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 23083 T731 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 23088 T731 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 23089 T731 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 23090 T731 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 23090 T731 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 23091 T731 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 23091 T731 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4]   2> 23092 T731 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 23092 T731 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty5-1363218079431/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty5/
[junit4:junit4]   2> 23092 T731 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@79024994
[junit4:junit4]   2> 23093 T731 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 23093 T731 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty5 forceNew: false
[junit4:junit4]   2> 23094 T731 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty5/index/
[junit4:junit4]   2> 23094 T731 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 23094 T731 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty5/index forceNew: false
[junit4:junit4]   2> 23097 T731 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@27d59293 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f63a81e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 23097 T731 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 23100 T731 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 23100 T731 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 23101 T731 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 23101 T731 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 23102 T731 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 23102 T731 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 23102 T731 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 23103 T731 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 23103 T731 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 23110 T731 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 23115 T731 oass.SolrIndexSearcher.<init> Opening Searcher@60c6fe68 main
[junit4:junit4]   2> 23116 T731 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty5/tlog
[junit4:junit4]   2> 23117 T731 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 23117 T731 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 23121 T732 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60c6fe68 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 23124 T731 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 23124 T731 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2>  C24_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:16138_, base_url=http://127.0.0.1:16138}
[junit4:junit4]   2> 23239 T714 C24 P16138 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:38368/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 23239 T714 C24 P16138 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:16138 START replicas=[http://127.0.0.1:38368/collection1/] nUpdates=100
[junit4:junit4]   2> 23241 T714 C24 P16138 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 23242 T714 C24 P16138 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 23242 T714 C24 P16138 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 23242 T714 C24 P16138 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 23243 T714 C24 P16138 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 23243 T714 C24 P16138 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:38368/collection1/. core=collection1
[junit4:junit4]   2> 23243 T714 C24 P16138 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 23244 T650 C22 P38368 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 23270 T649 C22 P38368 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 23271 T649 C22 P38368 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2cd39e6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4873daf9),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2cd39e6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4873daf9),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2cd39e6b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4873daf9),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 23272 T649 C22 P38368 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 23272 T649 C22 P38368 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23273 T649 C22 P38368 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 3
[junit4:junit4]   2> 23274 T714 C24 P16138 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 23274 T714 C24 P16138 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 23277 T652 C22 P38368 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 23278 T714 C24 P16138 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4]   2> 23279 T714 C24 P16138 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 23279 T714 C24 P16138 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 23281 T652 C22 P38368 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=0 
[junit4:junit4]   2> 23314 T714 C24 P16138 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 23316 T714 C24 P16138 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1363218059209/jetty4/index.20130313234122514 forceNew: false
[junit4:junit4]   2> 23316 T714 C24 P16138 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@4269b79 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f438ccd) fullCopy=false
[junit4:junit4]   2> 23320 T652 C22 P38368 REQ /replication {file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
[junit4:junit4]   2> 23322 T714 C24 P16138 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 23324 T714 C24 P16138 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 23325 T714 C24 P16138 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 23327 T714 C24 P16138 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5acc4401 lockFactory=org.apache.lucene.store.NativeFSLockFactory@367bbec2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5acc4401 lockFactory=org.apache.lucene.store.NativeFSLockFactory@367bbec2),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 23328 T714 C24 P16138 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 23328 T714 C24 P16138 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 23329 T714 C24 P16138 oass.SolrIndexSearcher.<init> Opening Searcher@1f2e3a27 main
[junit4:junit4]   2> 23330 T713 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f2e3a27 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4]   2> 23331 T714 C24 P16138 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@4269b79 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f438ccd)
[junit4:junit4]   2> 23331 T714 C24 P16138 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 23332 T714 C24 P16138 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 23332 T714 C24 P16138 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 23332 T714 C24 P16138 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 23354 T714 C24 P16138 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 23930 T640 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23931 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"6",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:49323_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:49323"}
[junit4:junit4]   2> 23932 T640 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 23932 T640 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 23937 T640 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:16138_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:16138"}
[junit4:junit4]   2> 23953 T639 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23953 T659 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23954 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23954 T646 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23954 T675 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23953 T711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 23953 T730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 24154 T731 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 24154 T731 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49323 collection:collection1 shard:shard1
[junit4:junit4]   2> 24162 T731 oasc.ZkController.register We are http://127.0.0.1:49323/collection1/ and leader is http://127.0.0.1:38368/collection1/
[junit4:junit4]   2> 24162 T731 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49323
[junit4:junit4]   2> 24162 T731 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 24163 T731 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C25 name=collection1 org.apache.solr.core.SolrCore@2405215f url=http://127.0.0.1:49323/collection1 node=127.0.0.1:49323_ C25_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49323_, base_url=http://127.0.0.1:49323}
[junit4:junit4]   2> 24163 T735 C25 P49323 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 24164 T735 C25 P49323 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 24165 T735 C25 P49323 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 24165 T731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 24165 T735 C25 P49323 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 24167 T617 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 24168 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 24169 T735 C25 P49323 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 24168 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 24452 T617 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 24456 T617 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:51772
[junit4:junit4]   2> 24457 T617 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 24458 T617 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 24459 T617 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty6-1363218083388
[junit4:junit4]   2> 24459 T617 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty6-1363218083388/solr.xml
[junit4:junit4]   2> 24460 T617 oasc.CoreContainer.<init> New CoreContainer 1730495579
[junit4:junit4]   2> 24461 T617 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty6-1363218083388/'
[junit4:junit4]   2> 24462 T617 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty6-1363218083388/'
[junit4:junit4]   2> 24543 T617 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 24544 T617 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 24545 T617 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 24545 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 24546 T617 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 24547 T617 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 24548 T617 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 24548 T617 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 24549 T617 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 24550 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 24587 T617 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 24618 T617 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35115/solr
[junit4:junit4]   2> 24619 T617 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 24620 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6d354203
[junit4:junit4]   2> 24622 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 24622 T746 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 24623 T746 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 24623 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:16612
[junit4:junit4]   2> 24624 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:16612
[junit4:junit4]   2> 24626 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd2000f with negotiated timeout 20000 for client /140.211.11.196:16612
[junit4:junit4]   2> 24626 T746 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35115, sessionid = 0x13d6620cfd2000f, negotiated timeout = 20000
[junit4:junit4]   2> 24627 T747 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d354203 name:ZooKeeperConnection Watcher:127.0.0.1:35115 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 24627 T617 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 24629 T622 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d6620cfd2000f
[junit4:junit4]   2> 24632 T747 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 24632 T619 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:16612 which had sessionid 0x13d6620cfd2000f
[junit4:junit4]   2> 24632 T617 oaz.ZooKeeper.close Session: 0x13d6620cfd2000f closed
[junit4:junit4]   2> 24633 T617 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 24653 T617 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35115/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@b9ad2d4
[junit4:junit4]   2> 24655 T617 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 24655 T748 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35115. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4]   2> 24656 T748 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35115, initiating session
[junit4:junit4]   2> 24656 T619 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:10242
[junit4:junit4]   2> 24657 T619 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:10242
[junit4:junit4]   2> 24659 T621 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d6620cfd20010 with negotiated timeout 20000 for client /140.211.11.196:10242
[junit4:junit4]   2> 246

[...truncated too long message...]

junit4]   2> 785703 T334 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/OpenCloseCoreStressTest_/00019_core/data
[junit4:junit4]   2> 785704 T334 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/OpenCloseCoreStressTest_/00019_core/data/index
[junit4:junit4]   2> 785704 T334 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/OpenCloseCoreStressTest_/00019_core/data/index
[junit4:junit4]   2> 785749 T334 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 786357 T334 oas.SolrTestCaseJ4.tearDown ###Ending test10MinutesNew
[junit4:junit4]   2> 786359 T334 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=el_GR, timezone=Australia/Lord_Howe
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Sun Microsystems Inc. 1.6.0_32 (64-bit)/cpus=16,threads=1,free=33589200,total=532873216
[junit4:junit4]   2> NOTE: All tests run in this JVM: [IndexBasedSpellCheckerTest, NoCacheHeaderTest, TestLMJelinekMercerSimilarityFactory, SortByFunctionTest, TestMultiCoreConfBootstrap, TestDefaultSimilarityFactory, UniqFieldsUpdateProcessorFactoryTest, TestLazyCores, DisMaxRequestHandlerTest, TestCollationKeyRangeQueries, DefaultValueUpdateProcessorTest, SolrPluginUtilsTest, LeaderElectionIntegrationTest, TestRecovery, RequestHandlersTest, TestCodecSupport, CurrencyFieldOpenExchangeTest, OpenCloseCoreStressTest]
[junit4:junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=OpenCloseCoreStressTest -Dtests.seed=2BE095131715C45A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=el_GR -Dtests.timezone=Australia/Lord_Howe -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR   0.00s J1 | OpenCloseCoreStressTest (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=949, name=qtp1882088204-949, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest], registration stack trace below.
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([2BE095131715C45A]:0)
[junit4:junit4]    > 	at java.lang.Thread.getStackTrace(Thread.java:1495)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:520)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:986)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:929)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:909)
[junit4:junit4]    > 	at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:904)
[junit4:junit4]    > 	at org.apache.solr.core.MockFSDirectoryFactory.create(MockFSDirectoryFactory.java:34)
[junit4:junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:272)
[junit4:junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:224)
[junit4:junit4]    > 	at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:124)
[junit4:junit4]    > 	at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1405)
[junit4:junit4]    > 	at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1541)
[junit4:junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:795)
[junit4:junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:619)
[junit4:junit4]    > 	at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:927)
[junit4:junit4]    > 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:957)
[junit4:junit4]    > 	at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1158)
[junit4:junit4]    > 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:226)
[junit4:junit4]    > 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
[junit4:junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
[junit4:junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
[junit4:junit4]    > 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4]    > 	at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:275)
[junit4:junit4]    > 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1074)
[junit4:junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
[junit4:junit4]    > 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4]    > 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
[junit4:junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4]    > 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4]    > 	at org.eclipse.jetty.server.Server.handle(Server.java:365)
[junit4:junit4]    > 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
[junit4:junit4]    > 	at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
[junit4:junit4]    > 	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
[junit4:junit4]    > 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
[junit4:junit4]    > 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:948)
[junit4:junit4]    > 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4]    > 	at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
[junit4:junit4]    > 	at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
[junit4:junit4]    > 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    > 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    > Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/OpenCloseCoreStressTest_/00016_core/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@727cc402)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	... 1 more
[junit4:junit4] Completed on J1 in 786.41s, 6 tests, 1 error, 2 skipped <<< FAILURES!

[...truncated 667 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:388: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:877: There were test failures: 272 suites, 1144 tests, 1 suite-level error, 1 failure, 26 ignored (7 assumptions)

Total time: 147 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message