lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 506 - Failure
Date Thu, 30 Jan 2014 17:46:30 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/506/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
Captured an uncaught exception in thread: Thread[id=935, name=Thread-537, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=935, name=Thread-537, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
	at __randomizedtesting.SeedInfo.seed([B198692402C4CA7B:307EE73C759BAA47]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: createcollection the collection time out:60s
	at __randomizedtesting.SeedInfo.seed([B198692402C4CA7B]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:495)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1621)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1649)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:920)




Build Log:
[...truncated 10183 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> 69476 T547 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /icwdx/oi
   [junit4]   2> 69481 T547 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./solrtest-CollectionsAPIDistributedZkTest-1391100339159
   [junit4]   2> 69492 T547 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 69500 T548 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 69701 T547 oasc.ZkTestServer.run start zk server on port:23319
   [junit4]   2> 69947 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 69989 T549 oazs.NIOServerCnxn.doIO WARN Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
   [junit4]   2> 71706 T552 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@472950a3 name:ZooKeeperConnection Watcher:127.0.0.1:23319 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 71706 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 71713 T547 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 71837 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 71840 T556 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e91d58b name:ZooKeeperConnection Watcher:127.0.0.1:23319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 71840 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 71848 T547 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 71854 T547 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 71860 T547 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 71864 T547 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 71869 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 71870 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 71886 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 71887 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 71893 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 71894 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 71899 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 71900 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 71905 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 71906 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 71911 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 71912 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 71917 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 71918 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 71923 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 71923 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 71928 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 71929 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 71934 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 71935 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 71940 T547 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 71941 T547 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> Using new style solr.xml
   [junit4]   2> 72337 T547 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 72341 T547 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:23325
   [junit4]   2> 72342 T547 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 72343 T547 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 72343 T547 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624
   [junit4]   2> 72344 T547 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/'
   [junit4]   2> 72398 T547 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/solr.xml
   [junit4]   2> 72509 T547 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 72510 T547 oasc.CoreContainer.<init> New CoreContainer 119193949
   [junit4]   2> 72511 T547 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/]
   [junit4]   2> 72513 T547 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 72513 T547 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 72514 T547 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 72514 T547 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 72515 T547 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 72515 T547 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 72516 T547 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 72516 T547 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 72517 T547 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 72530 T547 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 72531 T547 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 72531 T547 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 72532 T547 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:23319/solr
   [junit4]   2> 72580 T547 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 72581 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 72584 T567 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c719458 name:ZooKeeperConnection Watcher:127.0.0.1:23319 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 72585 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 72600 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 72602 T569 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1cdb6810 name:ZooKeeperConnection Watcher:127.0.0.1:23319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 72603 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 72618 T547 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 72641 T547 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 72666 T547 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 72669 T547 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:23325_icwdx%2Foi
   [junit4]   2> 72678 T547 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:23325_icwdx%2Foi
   [junit4]   2> 72684 T547 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 72688 T547 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 72716 T547 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:23325_icwdx%2Foi
   [junit4]   2> 72717 T547 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 72721 T547 oasc.Overseer.start Overseer (id=91167151852814339-127.0.0.1:23325_icwdx%2Foi-n_0000000000) starting
   [junit4]   2> 72758 T547 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 72783 T571 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 72783 T547 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 72787 T547 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 72791 T547 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 72815 T571 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 72821 T570 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 72829 T547 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624
   [junit4]   2> 73100 T547 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/conf/
   [junit4]   2> 73104 T547 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 73105 T547 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 73105 T547 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 73138 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 73142 T574 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77f175dc name:ZooKeeperConnection Watcher:127.0.0.1:23319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 73142 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 73144 T547 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 73151 T547 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 73451 T547 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 73452 T547 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 73455 T547 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:23333
   [junit4]   2> 73455 T547 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 73456 T547 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 73456 T547 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827
   [junit4]   2> 73457 T547 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/'
   [junit4]   2> 73496 T547 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/solr.xml
   [junit4]   2> 73600 T547 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 73601 T547 oasc.CoreContainer.<init> New CoreContainer 1979948697
   [junit4]   2> 73602 T547 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/]
   [junit4]   2> 73604 T547 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 73604 T547 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 73605 T547 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 73605 T547 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 73606 T547 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 73606 T547 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 73607 T547 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 73607 T547 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 73607 T547 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 73621 T547 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 73621 T547 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 73622 T547 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 73622 T547 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:23319/solr
   [junit4]   2> 73623 T547 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 73624 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 73628 T585 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67a18b56 name:ZooKeeperConnection Watcher:127.0.0.1:23319 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 73629 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 73650 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 73653 T587 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24c74cbe name:ZooKeeperConnection Watcher:127.0.0.1:23319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 73654 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 73661 T547 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 74668 T547 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:23333_icwdx%2Foi
   [junit4]   2> 74677 T547 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:23333_icwdx%2Foi
   [junit4]   2> 74689 T569 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 74690 T574 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 74690 T587 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 74705 T547 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827
   [junit4]   2> 74711 T547 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/conf/
   [junit4]   2> 74882 T547 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 74882 T547 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 74883 T547 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 75214 T547 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 75216 T547 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 75221 T547 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:23336
   [junit4]   2> 75222 T547 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 75222 T547 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 75223 T547 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560
   [junit4]   2> 75223 T547 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/'
   [junit4]   2> 75266 T547 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/solr.xml
   [junit4]   2> 75378 T547 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 75379 T547 oasc.CoreContainer.<init> New CoreContainer 1388755438
   [junit4]   2> 75380 T547 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/]
   [junit4]   2> 75381 T547 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 75382 T547 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 75382 T547 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 75383 T547 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 75383 T547 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 75384 T547 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 75384 T547 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 75384 T547 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 75385 T547 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 75395 T547 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 75396 T547 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 75396 T547 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 75397 T547 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:23319/solr
   [junit4]   2> 75397 T547 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 75398 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 75401 T599 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74d8e2b6 name:ZooKeeperConnection Watcher:127.0.0.1:23319 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 75402 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 75422 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 75425 T601 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3fc359ca name:ZooKeeperConnection Watcher:127.0.0.1:23319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 75425 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 75437 T547 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 76445 T547 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:23336_icwdx%2Foi
   [junit4]   2> 76447 T547 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:23336_icwdx%2Foi
   [junit4]   2> 76452 T574 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 76452 T569 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 76452 T601 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 76452 T587 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 76462 T547 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560
   [junit4]   2> 76742 T547 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/conf/
   [junit4]   2> 76744 T547 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 76744 T547 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 76745 T547 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 77044 T547 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 77045 T547 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 77048 T547 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:23342
   [junit4]   2> 77049 T547 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 77049 T547 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 77050 T547 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1391100346422
   [junit4]   2> 77050 T547 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1391100346422/'
   [junit4]   2> 77089 T547 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1391100346422/solr.xml
   [junit4]   2> 77174 T547 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 77175 T547 oasc.CoreContainer.<init> New CoreContainer 1502378669
   [junit4]   2> 77175 T547 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1391100346422/]
   [junit4]   2> 77176 T547 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 77177 T547 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 77177 T547 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 77177 T547 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 77178 T547 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 77178 T547 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 77178 T547 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 77179 T547 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 77179 T547 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 77186 T547 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 77186 T547 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 77187 T547 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 77187 T547 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:23319/solr
   [junit4]   2> 77187 T547 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 77188 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 77217 T613 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15ca21c9 name:ZooKeeperConnection Watcher:127.0.0.1:23319 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 77218 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 77221 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 77229 T615 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56b5b33e name:ZooKeeperConnection Watcher:127.0.0.1:23319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 77229 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 77235 T547 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 78240 T547 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:23342_icwdx%2Foi
   [junit4]   2> 78267 T547 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:23342_icwdx%2Foi
   [junit4]   2> 78272 T601 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 78272 T587 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 78272 T615 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 78272 T574 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 78272 T569 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 78282 T547 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1391100346422
   [junit4]   2> 78593 T547 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1391100346422/conf/
   [junit4]   2> 78604 T547 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 78605 T547 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 78605 T547 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 78926 T547 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 78927 T547 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 78930 T547 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:23345
   [junit4]   2> 78932 T547 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 78933 T547 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 78933 T547 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282
   [junit4]   2> 78934 T547 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/'
   [junit4]   2> 78978 T547 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/solr.xml
   [junit4]   2> 79087 T547 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 79088 T547 oasc.CoreContainer.<init> New CoreContainer 689233273
   [junit4]   2> 79089 T547 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/]
   [junit4]   2> 79091 T547 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 79091 T547 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 79092 T547 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 79092 T547 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 79093 T547 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 79093 T547 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 79094 T547 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 79094 T547 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 79095 T547 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 79108 T547 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 79108 T547 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 79109 T547 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 79109 T547 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:23319/solr
   [junit4]   2> 79110 T547 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 79111 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 79121 T627 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18d84801 name:ZooKeeperConnection Watcher:127.0.0.1:23319 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 79121 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 79125 T547 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 79134 T629 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2013c6b1 name:ZooKeeperConnection Watcher:127.0.0.1:23319/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 79135 T547 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 79142 T547 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 80148 T547 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:23345_icwdx%2Foi
   [junit4]   2> 80150 T547 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:23345_icwdx%2Foi
   [junit4]   2> 80155 T574 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 80155 T615 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 80155 T569 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 80155 T587 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 80155 T601 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 80155 T629 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 80166 T547 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282
   [junit4]   2> 80169 T547 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/conf/
   [junit4]   2> 80455 T547 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 80456 T547 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 80457 T547 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 80524 T577 oasha.CollectionsHandler.handleCreateAction Creating Collection : name=solrj_collection&action=CREATE&collection.configName=conf1&router.field=myOwnField&wt=javabin&replicationFactor=2&numShards=2&version=2&router.name=compositeId
   [junit4]   2> 80528 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 80553 T571 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField",
   [junit4]   2> 	  "router.name":"compositeId"}
   [junit4]   2> 80553 T571 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField",
   [junit4]   2> 	  "router.name":"compositeId"}
   [junit4]   2> 80554 T571 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection 
   [junit4]   2> 80554 T571 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
   [junit4]   2> 80592 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 80594 T570 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 80595 T570 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
   [junit4]   2> 80596 T570 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
   [junit4]   2> 80631 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 80647 T569 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80647 T574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80647 T629 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80647 T615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80647 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80647 T601 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80693 T571 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2] , repFactor : 2
   [junit4]   2> 80693 T571 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:23345_icwdx%2Foi
   [junit4]   2> 80694 T571 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:23336_icwdx%2Foi
   [junit4]   2> 80695 T571 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:23333_icwdx%2Foi
   [junit4]   2> 80695 T571 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:23325_icwdx%2Foi
   [junit4]   2> 80699 T578 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down
   [junit4]   2> 80700 T559 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down
   [junit4]   2> 80700 T591 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down
   [junit4]   2> 80700 T619 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down
   [junit4]   2> 80702 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 80702 T578 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 80703 T559 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 80703 T591 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 80713 T619 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 80715 T570 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 80716 T570 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:23333/icwdx/oi",
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:23333_icwdx%2Foi",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 80727 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 80728 T570 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:23325/icwdx/oi",
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:23325_icwdx%2Foi",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 80733 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 80735 T570 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:23336/icwdx/oi",
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:23336_icwdx%2Foi",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 80744 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 80746 T570 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:23345/icwdx/oi",
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:23345_icwdx%2Foi",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 80751 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 80856 T574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80856 T601 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80856 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80856 T569 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80856 T629 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 80856 T615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 81704 T578 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
   [junit4]   2> 81704 T559 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
   [junit4]   2> 81705 T591 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
   [junit4]   2> 81707 T578 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using instanceDir: ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/solrj_collection_shard2_replica1
   [junit4]   2> 81707 T578 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 81708 T559 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/solrj_collection_shard2_replica2
   [junit4]   2> 81708 T559 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 81709 T578 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 81708 T591 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/solrj_collection_shard1_replica2
   [junit4]   2> 81709 T559 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 81710 T591 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 81709 T578 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 81710 T559 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 81711 T591 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 81711 T591 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 81713 T578 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/solrj_collection_shard2_replica1/'
   [junit4]   2> 81713 T559 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/solrj_collection_shard2_replica2/'
   [junit4]   2> 81713 T591 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/solrj_collection_shard1_replica2/'
   [junit4]   2> 81714 T619 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
   [junit4]   2> 81718 T619 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/solrj_collection_shard1_replica1
   [junit4]   2> 81718 T619 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 81719 T619 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 81719 T619 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 81721 T619 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/solrj_collection_shard1_replica1/'
   [junit4]   2> 81877 T578 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 81877 T591 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 81879 T559 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 81879 T619 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 81977 T591 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 81977 T578 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 81978 T559 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 81980 T619 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 81980 T591 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 81980 T559 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 81980 T578 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 81982 T619 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 82006 T591 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
   [junit4]   2> 82006 T578 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
   [junit4]   2> 82008 T559 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
   [junit4]   2> 82009 T619 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
   [junit4]   2> 82811 T591 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 82812 T559 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 82818 T619 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 82826 T591 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 82827 T559 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 82828 T619 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 82831 T591 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 82832 T559 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 82833 T619 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 82842 T578 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 82852 T578 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 82857 T578 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 82867 T591 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 82867 T619 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 82868 T559 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 82875 T619 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 82875 T559 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 82875 T591 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 82881 T619 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 82882 T559 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 82883 T619 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 82885 T619 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 82885 T559 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 82886 T591 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 82886 T559 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 82886 T619 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 82888 T559 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 82889 T591 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 82890 T619 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 82890 T619 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 82890 T591 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 82891 T619 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 82890 T559 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 82892 T619 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/solrj_collection_shard1_replica1/, dataDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/solrj_collection_shard1_replica1/data/
   [junit4]   2> 82893 T578 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 82891 T591 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 82893 T619 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b685e12
   [junit4]   2> 82892 T559 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 82895 T559 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 82896 T559 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/solrj_collection_shard2_replica2/, dataDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/solrj_collection_shard2_replica2/data/
   [junit4]   2> 82896 T591 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 82896 T559 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b685e12
   [junit4]   2> 82897 T591 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 82898 T591 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 82898 T591 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/solrj_collection_shard1_replica2/, dataDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/solrj_collection_shard1_replica2/data/
   [junit4]   2> 82899 T591 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b685e12
   [junit4]   2> 82900 T578 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 82908 T619 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/solrj_collection_shard1_replica1/data
   [junit4]   2> 82908 T591 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/solrj_collection_shard1_replica2/data
   [junit4]   2> 82908 T619 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/solrj_collection_shard1_replica1/data/index/
   [junit4]   2> 82908 T559 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/solrj_collection_shard2_replica2/data
   [junit4]   2> 82909 T619 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 82909 T578 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 82909 T591 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/solrj_collection_shard1_replica2/data/index/
   [junit4]   2> 82910 T619 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1391100348282/solrj_collection_shard1_replica1/data/index
   [junit4]   2> 82909 T559 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/solrj_collection_shard2_replica2/data/index/
   [junit4]   2> 82912 T619 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=63.4228515625, floorSegmentMB=0.9013671875, forceMergeDeletesPctAllowed=14.367443874286256, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7708475508539009
   [junit4]   2> 82911 T591 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 82912 T578 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 82912 T559 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 82913 T578 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 82913 T619 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6c7179a0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6d385084),segFN=segments_1,generation=1}
   [junit4]   2> 82913 T591 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1391100344560/solrj_collection_shard1_replica2/data/index
   [junit4]   2> 82914 T619 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 82914 T578 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 82914 T559 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1391100341624/solrj_collection_shard2_replica2/data/index
   [junit4]   2> 82915 T591 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=63.4228515625, floorSegmentMB=0.9013671875, forceMergeDeletesPctAllowed=14.367443874286256, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7708475508539009
   [junit4]   2> 82917 T559 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=63.4228515625, floorSegmentMB=0.9013671875, forceMergeDeletesPctAllowed=14.367443874286256, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7708475508539009
   [junit4]   2> 82918 T591 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@30f78ac6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13e4893c),segFN=segments_1,generation=1}
   [junit4]   2> 82918 T559 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@7c02a5e1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@373cb761),segFN=segments_1,generation=1}
   [junit4]   2> 82918 T578 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 82919 T559 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 82918 T591 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 82919 T578 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 82921 T578 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 82922 T578 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/solrj_collection_shard2_replica1/, dataDir=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/solrj_collection_shard2_replica1/data/
   [junit4]   2> 82923 T578 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b685e12
   [junit4]   2> 82924 T578 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/solrj_collection_shard2_replica1/data
   [junit4]   2> 82925 T578 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/solrj_collection_shard2_replica1/data/index/
   [junit4]   2> 82925 T578 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory './org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 82926 T578 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1391100342827/solrj_collection_shard2_replica1/data/index
   [junit4]   2> 82927 T578 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=63.4228515625, floorSegmentMB=0.9013671875, forceMergeDeletesPctAllowed=14.367443874286256, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7708475508539009
   [junit4]   2> 82928 T578 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3d08a9e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@32bb8931),segFN=segments_1,generation=1}
   [junit4]   2> 82928 T578 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 82939 T559 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 82939 T619 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 82940 T578 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 82940 T559 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 82939 T591 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 82941 T559 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 82941 T578 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 82940 T619 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 82942 T578 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 82942 T559 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 82942 T591 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 82943 T559 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 82943 T578 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 82943 T619 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 82945 T578 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 82944 T559 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 82944 T591 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 82945 T578 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 82945 T619 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 82947 T578 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 82947 T619 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 82947 T559 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 82946 T591 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 82948 T559 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 82948 T619 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 82948 T578 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 82950 T619 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 82950 T578 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 82949 T559 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 82949 T591 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 82951 T619 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 82952 T578 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 82952 T591 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 82952 T559 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 82953 T591 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 82952 T619 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 82954 T578 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 82955 T578 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 82954 T591 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 82956 T578 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 82956 T619 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 82954 T559 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 82957 T559 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 82956 T591 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 82958 T559 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 82958 T619 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 82959 T619 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 82957 T578 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 82960 T619 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 82960 T578 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 82959 T559 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 82959 T591 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 82961 T559 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 82961 T619 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 82962 T578 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 82963 T619 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 82963 T559 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 82963 T591 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 82964 T591 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 82964 T619 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 82965 T591 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 82965 T559 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 82964 T578 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 82966 T619 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 82966 T591 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 82967 T591 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 82968 T591 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 82970 T591 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 83003 T591 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 83004 T559 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 83004 T578 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 83004 T619 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 83013 T578 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 83014 T591 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 83014 T559 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 83014 T578 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 83015 T559 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 83015 T619 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 83015 T591 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 83017 T619 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 83018 T578 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@788ffbe9
   [junit4]   2> 83018 T559 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@34e3bbd5
   [junit4]   2> 83019 T619 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@205cd1d
   [junit4]   2> 83019 T591 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3df51218
   [junit4]   2> 83019 T559 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@7c02a5e1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@373cb761),segFN=segments_1,generation=1}
   [junit4]   2> 83020 T591 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@30f78ac6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13e4893c),segFN=segments_1,generation=1}
   [junit4]   2> 83019 T578 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3d08a9e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@32bb8931),segFN=segments_1,generation=1}
   [junit4]   2> 83021 T591 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 83020 T559 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 83020 T619 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6c7179a0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6d385084),segFN=segments_1,generation=1}
   [junit4]   2> 83023 T559 oass.SolrIndexSearcher.<init> Opening Searcher@3be4c961 main
   [junit4]   2> 83022 T591 oass.SolrIndexSearcher.<init> Opening Searcher@5d18daf6 main
   [junit4]   2> 83021 T578 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 83023 T619 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 83024 T578 oass.SolrIndexSearcher.<init> Opening Searcher@4d45094f main
   [junit4]   2> 83025 T619 oass.SolrIndexSearcher.<init> Opening Searcher@244d9d89 main
   [junit4]   2> 83035 T635 oasc.SolrCore.registerSearcher [solrj_collection_shard1_replica2] Registered new searcher Searcher@5d18daf6 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 83037 T638 oasc.SolrCore.registerSearcher [solrj_collection_shard1_replica1] Registered new searcher Searcher@244d9d89 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 83037 T636 oasc.SolrCore.registerSearcher [solrj_collection_shard2_replica2] Registered new searcher Searcher@3be4c961 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 83046 T637 oasc.SolrCore.registerSearcher [solrj_collection_shard2_replica1] Registered new searcher Searcher@4d45094f main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 83054 T591 oasc.CoreContainer.registerCore registering core: solrj_collection_shard1_replica2
   [junit4]   2> 83055 T591 oasc.ZkController.register Register replica - core:solrj_collection_shard1_replica2 address:http://127.0.0.1:23336/icwdx/oi collection:solrj_collection shard:shard1
   [junit4]   2> 83056 T559 oasc.CoreContainer.registerCore registering core: solrj_collection_shard2_replica2
   [junit4]   2> 83056 T619 oasc.CoreContainer.registerCore registering core: solrj_collection_shard1_replica1
   [junit4]   2> 83057 T559 oasc.ZkController.register Register replica - core:solrj_collection_shard2_replica2 address:http://127.0.0.1:23325/icwdx/oi collection:solrj_collection shard:shard2
   [junit4]   2> 83057 T619 oasc.ZkController.register Register replica - core:solrj_collection_shard1_replica1 address:http://127.0.0.1:23345/icwdx/oi collection:solrj_collection shard:shard1
   [junit4]   2> 83063 T578 oasc.CoreContainer.registerCore registering core: solrj_collection_shard2_replica1
   [junit4]   2> 83064 T578 oasc.ZkController.register Register replica - core:solrj_collection_shard2_replica1 address:http://127.0.0.1:23333/icwdx/oi collection:solrj_collection shard:shard2
   [junit4]   2> 83078 T578 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard2/election
   [junit4]   2> 83078 T591 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard1/election
   [junit4]   2> 83078 T619 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard1/election
   [junit4]   2> 83078 T559 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard2/election
   [junit4]   2> 83114 T559 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 83115 T591 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 83129 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 83130 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 83130 T569 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 83136 T559 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 83137 T591 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 83137 T559 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 83138 T591 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 83139 T570 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> ASYNC  NEW_CORE C52 name=solrj_collection_shard1_replica2 org.apache.solr.core.SolrCore@464786d4 url=http://127.0.0.1:23336/icwdx/oi/solrj_collection_shard1_replica2 node=127.0.0.1:23336_icwdx%2Foi C52_STATE=coll:solrj_collection core:solrj_collection_shard1_replica2 props:{state=down, base_url=http://127.0.0.1:23336/icwdx/oi, core=solrj_collection_shard1_replica2, node_name=127.0.0.1:23336_icwdx%2Foi}
   [junit4]   2> 83146 T591 C52 P23336 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:23336/icwdx/oi/solrj_collection_shard1_replica2/
   [junit4]   2> ASYNC  NEW_CORE C53 name=solrj_collection_shard2_replica2 org.apache.solr.core.SolrCore@3185c8e2 url=http://127.0.0.1:23325/icwdx/oi/solrj_collection_s

[...truncated too long message...]

256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 606331 T629 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewcollection_2/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 606332 T629 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewstresscollection_collection5_8/leader_elect/shard3/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 606332 T629 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewstresscollection_collection5_8/leader_elect/shard6/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 606333 T629 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewstresscollection_collection5_8/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 606333 T629 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=453), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=453)), sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_LY, timezone=Zulu
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=3,free=54717440,total=504102912
   [junit4]   2> NOTE: All tests run in this JVM: [BadIndexSchemaTest, SuggesterWFSTTest, TestLazyCores, TestAtomicUpdateErrorCases, TestDistributedMissingSort, SuggesterTSTTest, JSONWriterTest, TestSolrCoreProperties, TestUpdate, UniqFieldsUpdateProcessorFactoryTest, TestFieldTypeCollectionResource, CollectionsAPIDistributedZkTest]
   [junit4] Completed on J1 in 537.60s, 1 test, 1 error <<< FAILURES!

[...truncated 939 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:460: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:433: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:491: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1268: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:901: There were test failures: 365 suites, 1604 tests, 1 error, 33 ignored (5 assumptions)

Total time: 136 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message