lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4514 - Still Failing
Date Mon, 02 Dec 2013 04:34:25 GMT
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4514/

1 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.testDistribSearch

Error Message:
There were expected update fails expected:<0> but was:<1>

Stack Trace:
java.lang.AssertionError: There were expected update fails expected:<0> but was:<1>
	at __randomizedtesting.SeedInfo.seed([AECDF85261C18C80:2F2B764A169EECBC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:202)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:843)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:724)




Build Log:
[...truncated 10365 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
   [junit4]   2> 697610 T748 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 697623 T748 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyNothingIsSafeTest-1385956797376
   [junit4]   2> 697625 T748 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 697626 T749 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 697726 T748 oasc.ZkTestServer.run start zk server on port:13165
   [junit4]   2> 697727 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 697733 T755 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1244e56 name:ZooKeeperConnection Watcher:127.0.0.1:13165 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 697733 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 697734 T748 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 697740 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 697760 T757 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f8dc872 name:ZooKeeperConnection Watcher:127.0.0.1:13165/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 697761 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 697761 T748 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 697810 T748 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 697814 T748 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 697855 T748 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 697860 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 697871 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 697949 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 697959 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 698066 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 698067 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 698071 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 698072 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 698077 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 698078 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 698087 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 698088 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 698095 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 698096 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 698114 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 698115 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 698119 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 698120 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 698129 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 698129 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 698145 T748 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 698146 T748 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 698520 T748 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 698525 T748 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:62288
   [junit4]   2> 698526 T748 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 698526 T748 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 698527 T748 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385956797904
   [junit4]   2> 698527 T748 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385956797904/'
   [junit4]   2> 698569 T748 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385956797904/solr.xml
   [junit4]   2> 698679 T748 oasc.CoreContainer.<init> New CoreContainer 967037106
   [junit4]   2> 698680 T748 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385956797904/]
   [junit4]   2> 698682 T748 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 698682 T748 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 698683 T748 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 698683 T748 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 698684 T748 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 698684 T748 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 698684 T748 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 698685 T748 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 698685 T748 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 698706 T748 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 698706 T748 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 698707 T748 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 698707 T748 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:13165/solr
   [junit4]   2> 698708 T748 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 698709 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 698712 T768 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3826a29b name:ZooKeeperConnection Watcher:127.0.0.1:13165 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 698713 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 698716 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 698729 T770 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b87a5a1 name:ZooKeeperConnection Watcher:127.0.0.1:13165/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 698729 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 698732 T748 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 698738 T748 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 698742 T748 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 698749 T748 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62288_
   [junit4]   2> 698751 T748 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:62288_
   [junit4]   2> 698755 T748 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 698773 T748 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 698776 T748 oasc.Overseer.start Overseer (id=90830064673357827-127.0.0.1:62288_-n_0000000000) starting
   [junit4]   2> 698787 T748 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 698800 T772 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 698801 T748 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 698804 T748 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 698807 T748 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 698811 T771 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 698817 T773 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 698817 T773 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 698819 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 698819 T773 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 698821 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 698822 T771 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:62288",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:62288_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 698823 T771 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 698823 T771 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 698826 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 698835 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 699821 T773 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 699821 T773 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385956797904/collection1
   [junit4]   2> 699821 T773 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 699822 T773 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 699823 T773 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 699824 T773 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385956797904/collection1/'
   [junit4]   2> 699825 T773 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385956797904/collection1/lib/README' to classloader
   [junit4]   2> 699826 T773 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385956797904/collection1/lib/classes/' to classloader
   [junit4]   2> 699892 T773 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 699955 T773 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 700058 T773 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 700069 T773 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 700677 T773 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 700681 T773 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 700684 T773 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 700690 T773 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 700725 T773 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 700781 T773 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-controljetty-1385956797904/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/control/data/
   [junit4]   2> 700782 T773 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a2f0d89
   [junit4]   2> 700783 T773 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/control/data
   [junit4]   2> 700784 T773 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/control/data/index/
   [junit4]   2> 700785 T773 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 700787 T773 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/control/data/index
   [junit4]   2> 700788 T773 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=65.7900390625, floorSegmentMB=0.6103515625, forceMergeDeletesPctAllowed=4.994081044040208, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6507784844660305
   [junit4]   2> 700812 T773 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 700813 T773 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 700817 T773 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 700818 T773 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 700818 T773 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 700818 T773 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 700819 T773 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 700819 T773 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 700819 T773 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 700820 T773 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 700820 T773 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 700821 T773 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 700821 T773 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 700821 T773 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 700822 T773 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 700822 T773 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 700823 T773 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 700823 T773 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 700840 T773 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 700844 T773 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 700844 T773 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 700845 T773 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 700847 T773 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 700848 T773 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 700848 T773 oass.SolrIndexSearcher.<init> Opening Searcher@6f028b1c main
   [junit4]   2> 700852 T774 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f028b1c main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 700853 T773 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 700854 T773 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:62288 collection:control_collection shard:shard1
   [junit4]   2> 700855 T773 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 700882 T773 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 700896 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 700896 T773 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 700897 T773 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C324 name=collection1 org.apache.solr.core.SolrCore@7d051018 url=http://127.0.0.1:62288/collection1 node=127.0.0.1:62288_ C324_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:62288, core=collection1, node_name=127.0.0.1:62288_}
   [junit4]   2> 700897 T773 C324 P62288 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:62288/collection1/
   [junit4]   2> 700897 T773 C324 P62288 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 700897 T773 C324 P62288 oasc.SyncStrategy.syncToMe http://127.0.0.1:62288/collection1/ has no replicas
   [junit4]   2> 700898 T773 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:62288/collection1/ shard1
   [junit4]   2> 700898 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 700898 T773 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 700918 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 700920 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 700953 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 700968 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 701072 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 701107 T773 oasc.ZkController.register We are http://127.0.0.1:62288/collection1/ and leader is http://127.0.0.1:62288/collection1/
   [junit4]   2> 701107 T773 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:62288
   [junit4]   2> 701107 T773 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 701108 T773 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 701108 T773 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 701110 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 701111 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 701111 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 701111 T773 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 701113 T748 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 701113 T748 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 701113 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 701114 T771 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:62288",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:62288_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 701126 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 701134 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 701136 T777 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3774304c name:ZooKeeperConnection Watcher:127.0.0.1:13165/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 701136 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 701138 T748 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 701141 T748 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 701239 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 701239 T777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 701547 T748 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 701551 T748 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:46819
   [junit4]   2> 701551 T748 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 701552 T748 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 701552 T748 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385956800893
   [junit4]   2> 701553 T748 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385956800893/'
   [junit4]   2> 701614 T748 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385956800893/solr.xml
   [junit4]   2> 701748 T748 oasc.CoreContainer.<init> New CoreContainer 1671859949
   [junit4]   2> 701749 T748 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385956800893/]
   [junit4]   2> 701751 T748 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 701751 T748 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 701752 T748 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 701752 T748 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 701752 T748 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 701753 T748 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 701753 T748 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 701754 T748 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 701754 T748 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 701775 T748 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 701776 T748 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 701776 T748 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 701777 T748 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:13165/solr
   [junit4]   2> 701777 T748 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 701778 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 701781 T788 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f342fe8 name:ZooKeeperConnection Watcher:127.0.0.1:13165 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 701782 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 701800 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 701809 T790 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a6a0d4e name:ZooKeeperConnection Watcher:127.0.0.1:13165/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 701809 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 701838 T748 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 702842 T748 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46819_
   [junit4]   2> 702846 T748 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46819_
   [junit4]   2> 702913 T777 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 702913 T790 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 702913 T770 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 702936 T791 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 702937 T791 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 702973 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702974 T791 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 702974 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702974 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 702975 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 702976 T771 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:46819",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:46819_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 702977 T771 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 702977 T771 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 703132 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 703146 T790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 703146 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 703146 T777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 703975 T791 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 703975 T791 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385956800893/collection1
   [junit4]   2> 703975 T791 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 703977 T791 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 703977 T791 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 703978 T791 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385956800893/collection1/'
   [junit4]   2> 703980 T791 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385956800893/collection1/lib/README' to classloader
   [junit4]   2> 703980 T791 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385956800893/collection1/lib/classes/' to classloader
   [junit4]   2> 704048 T791 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 704142 T791 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 704245 T791 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 704257 T791 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 704993 T791 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 704998 T791 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 705000 T791 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 705006 T791 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 705041 T791 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 705041 T791 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty1-1385956800893/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty1/
   [junit4]   2> 705041 T791 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a2f0d89
   [junit4]   2> 705043 T791 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty1
   [junit4]   2> 705044 T791 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty1/index/
   [junit4]   2> 705045 T791 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 705046 T791 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty1/index
   [junit4]   2> 705047 T791 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=65.7900390625, floorSegmentMB=0.6103515625, forceMergeDeletesPctAllowed=4.994081044040208, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6507784844660305
   [junit4]   2> 705054 T791 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 705055 T791 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 705061 T791 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 705062 T791 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 705062 T791 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 705062 T791 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 705063 T791 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 705063 T791 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 705063 T791 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 705064 T791 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 705064 T791 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 705065 T791 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 705066 T791 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 705066 T791 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 705066 T791 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 705067 T791 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 705067 T791 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 705068 T791 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 705088 T791 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 705092 T791 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 705093 T791 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 705094 T791 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 705096 T791 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 705096 T791 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 705097 T791 oass.SolrIndexSearcher.<init> Opening Searcher@702b5459 main
   [junit4]   2> 705102 T792 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@702b5459 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 705105 T791 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 705105 T791 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:46819 collection:collection1 shard:shard1
   [junit4]   2> 705107 T791 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 705132 T791 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 705136 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705136 T791 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 705136 T791 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C325 name=collection1 org.apache.solr.core.SolrCore@d4f8900 url=http://127.0.0.1:46819/collection1 node=127.0.0.1:46819_ C325_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:46819, core=collection1, node_name=127.0.0.1:46819_}
   [junit4]   2> 705137 T791 C325 P46819 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:46819/collection1/
   [junit4]   2> 705137 T791 C325 P46819 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 705137 T791 C325 P46819 oasc.SyncStrategy.syncToMe http://127.0.0.1:46819/collection1/ has no replicas
   [junit4]   2> 705137 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 705137 T791 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:46819/collection1/ shard1
   [junit4]   2> 705138 T791 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 705148 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705155 T777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705155 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705155 T790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705175 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 705180 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705283 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705283 T777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705283 T790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705328 T791 oasc.ZkController.register We are http://127.0.0.1:46819/collection1/ and leader is http://127.0.0.1:46819/collection1/
   [junit4]   2> 705328 T791 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46819
   [junit4]   2> 705328 T791 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 705328 T791 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 705329 T791 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 705331 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705331 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705331 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705331 T791 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 705333 T748 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 705333 T748 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 705333 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 705334 T771 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:46819",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:46819_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 705348 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 705451 T790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705451 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705451 T777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 705697 T748 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 705705 T748 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56134
   [junit4]   2> 705705 T748 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 705706 T748 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 705706 T748 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385956805087
   [junit4]   2> 705707 T748 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385956805087/'
   [junit4]   2> 705752 T748 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385956805087/solr.xml
   [junit4]   2> 705847 T748 oasc.CoreContainer.<init> New CoreContainer 547503238
   [junit4]   2> 705848 T748 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385956805087/]
   [junit4]   2> 705850 T748 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 705850 T748 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 705851 T748 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 705851 T748 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 705852 T748 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 705852 T748 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 705853 T748 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 705853 T748 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 705853 T748 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 705873 T748 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 705873 T748 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 705874 T748 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 705875 T748 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:13165/solr
   [junit4]   2> 705875 T748 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 705876 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 705883 T804 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4aaf6317 name:ZooKeeperConnection Watcher:127.0.0.1:13165 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 705884 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 705888 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 705902 T806 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@704c5ae3 name:ZooKeeperConnection Watcher:127.0.0.1:13165/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 705903 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 705916 T748 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 706920 T748 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56134_
   [junit4]   2> 706923 T748 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56134_
   [junit4]   2> 706927 T770 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 706927 T806 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 706927 T790 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 706927 T777 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 706945 T807 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 706945 T807 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 706947 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706947 T807 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 706947 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706947 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706949 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 706950 T771 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56134",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:56134_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 706950 T771 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 706951 T771 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 706954 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 706963 T790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 706963 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 706963 T777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 706963 T806 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 707948 T807 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 707948 T807 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385956805087/collection1
   [junit4]   2> 707948 T807 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 707950 T807 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 707950 T807 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 707951 T807 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385956805087/collection1/'
   [junit4]   2> 707953 T807 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385956805087/collection1/lib/README' to classloader
   [junit4]   2> 707953 T807 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385956805087/collection1/lib/classes/' to classloader
   [junit4]   2> 708078 T807 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 708163 T807 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 708266 T807 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 708279 T807 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 709020 T807 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 709024 T807 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 709027 T807 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 709033 T807 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 709066 T807 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 709067 T807 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty2-1385956805087/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty2/
   [junit4]   2> 709067 T807 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a2f0d89
   [junit4]   2> 709069 T807 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty2
   [junit4]   2> 709070 T807 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty2/index/
   [junit4]   2> 709071 T807 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 709072 T807 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty2/index
   [junit4]   2> 709073 T807 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=65.7900390625, floorSegmentMB=0.6103515625, forceMergeDeletesPctAllowed=4.994081044040208, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6507784844660305
   [junit4]   2> 709078 T807 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty2/index,segFN=segments_1,generation=1}
   [junit4]   2> 709078 T807 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 709084 T807 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 709085 T807 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 709085 T807 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 709085 T807 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 709086 T807 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 709086 T807 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 709087 T807 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 709087 T807 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 709087 T807 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 709088 T807 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 709089 T807 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 709089 T807 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 709090 T807 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 709090 T807 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 709091 T807 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 709092 T807 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 709111 T807 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 709116 T807 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 709116 T807 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 709117 T807 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 709125 T807 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty2/index,segFN=segments_1,generation=1}
   [junit4]   2> 709126 T807 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 709126 T807 oass.SolrIndexSearcher.<init> Opening Searcher@49a3cbde main
   [junit4]   2> 709131 T808 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@49a3cbde main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 709135 T807 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 709135 T807 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56134 collection:collection1 shard:shard2
   [junit4]   2> 709136 T807 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 709161 T807 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 709172 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 709172 T807 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 709173 T807 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C326 name=collection1 org.apache.solr.core.SolrCore@37faacf url=http://127.0.0.1:56134/collection1 node=127.0.0.1:56134_ C326_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:56134, core=collection1, node_name=127.0.0.1:56134_}
   [junit4]   2> 709173 T807 C326 P56134 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56134/collection1/
   [junit4]   2> 709173 T807 C326 P56134 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 709174 T807 C326 P56134 oasc.SyncStrategy.syncToMe http://127.0.0.1:56134/collection1/ has no replicas
   [junit4]   2> 709174 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 709174 T807 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56134/collection1/ shard2
   [junit4]   2> 709174 T807 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 709184 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 709185 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709186 T790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709185 T777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709185 T806 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709199 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 709203 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 709307 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709307 T806 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709307 T790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709307 T777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709351 T807 oasc.ZkController.register We are http://127.0.0.1:56134/collection1/ and leader is http://127.0.0.1:56134/collection1/
   [junit4]   2> 709351 T807 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56134
   [junit4]   2> 709351 T807 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 709351 T807 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 709352 T807 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 709354 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 709354 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 709354 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 709354 T807 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 709356 T748 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 709357 T748 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 709356 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 709358 T771 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56134",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:56134_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 709377 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 709480 T790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709480 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709480 T777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709480 T806 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 709781 T748 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 709785 T748 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65508
   [junit4]   2> 709785 T748 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 709786 T748 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 709786 T748 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385956809110
   [junit4]   2> 709787 T748 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385956809110/'
   [junit4]   2> 709838 T748 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385956809110/solr.xml
   [junit4]   2> 709976 T748 oasc.CoreContainer.<init> New CoreContainer 1095095705
   [junit4]   2> 709976 T748 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385956809110/]
   [junit4]   2> 709978 T748 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 709979 T748 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 709979 T748 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 709979 T748 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 709980 T748 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 709980 T748 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 709981 T748 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 709981 T748 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 709982 T748 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 710002 T748 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 710002 T748 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 710003 T748 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 710003 T748 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:13165/solr
   [junit4]   2> 710004 T748 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 710005 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 710024 T820 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a364a7b name:ZooKeeperConnection Watcher:127.0.0.1:13165 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 710025 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 710046 T748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 710061 T822 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53fbaad3 name:ZooKeeperConnection Watcher:127.0.0.1:13165/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 710061 T748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 710083 T748 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 711087 T748 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65508_
   [junit4]   2> 711090 T748 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65508_
   [junit4]   2> 711094 T777 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 711094 T770 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 711094 T806 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 711094 T822 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 711094 T790 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 711113 T823 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 711114 T823 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 711115 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 711115 T823 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 711116 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 711116 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 711118 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 711119 T771 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:65508",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:65508_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 711119 T771 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 711119 T771 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 711137 T770 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 711138 T777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 711139 T806 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 711139 T790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 711139 T822 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 711138 T770 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 712117 T823 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 712117 T823 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385956809110/collection1
   [junit4]   2> 712117 T823 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 712118 T823 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 712119 T823 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 712120 T823 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385956809110/collection1/'
   [junit4]   2> 712121 T823 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385956809110/collection1/lib/README' to classloader
   [junit4]   2> 712122 T823 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385956809110/collection1/lib/classes/' to classloader
   [junit4]   2> 712202 T823 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 712274 T823 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 712376 T823 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 712388 T823 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 713129 T823 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 713133 T823 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 713136 T823 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 713160 T823 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 713194 T823 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 713194 T823 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-jetty3-1385956809110/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty3/
   [junit4]   2> 713195 T823 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a2f0d89
   [junit4]   2> 713199 T823 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty3
   [junit4]   2> 713200 T823 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty3/index/
   [junit4]   2> 713201 T823 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty3/index' doesn't exist. Creating new index...
   [junit4]   2> 713202 T823 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty3/index
   [junit4]   2> 713204 T823 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=65.7900390625, floorSegmentMB=0.6103515625, forceMergeDeletesPctAllowed=4.994081044040208, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6507784844660305
   [junit4]   2> 713212 T823 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest-1385956797375/jetty3/index,segFN=segments_1,generation=1}
   [junit4]   2> 713212 T823 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 713219 T823 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 713234 T823 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 713234 T823 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 713235 T823 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 713235 T823 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 713235 T823 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 713236 T823 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 713236 T823 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 713236 T823 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 713237 T823 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 713238 T823 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 713238 T823 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 713239 T823 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 713239 T823 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 713240 T823 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 713241 T823 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 713263 T823 oashl.XMLLoader.init 

[...truncated too long message...]

hards":{"shard1":{
   [junit4]   1>               "range":"80000000-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{"core_node1":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:62288",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:62288_",
   [junit4]   1>                   "leader":"true"}}}}},
   [junit4]   1>         "collection1":{
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "shards":{
   [junit4]   1>             "shard1":{
   [junit4]   1>               "range":"80000000-ffffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node1":{
   [junit4]   1>                   "state":"down",
   [junit4]   1>                   "base_url":"http://127.0.0.1:46819",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:46819_"},
   [junit4]   1>                 "core_node3":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:65508",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:65508_"},
   [junit4]   1>                 "core_node5":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:34156",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:34156_",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node7":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:57103",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:57103_"}}},
   [junit4]   1>             "shard2":{
   [junit4]   1>               "range":"0-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node2":{
   [junit4]   1>                   "state":"down",
   [junit4]   1>                   "base_url":"http://127.0.0.1:56134",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:56134_"},
   [junit4]   1>                 "core_node4":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:53793",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:53793_"},
   [junit4]   1>                 "core_node6":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:53222",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:53222_",
   [junit4]   1>                   "leader":"true"}}}}}}
   [junit4]   1>   /solr/live_nodes (6)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:57103_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:65508_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:62288_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:34156_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:53222_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:53793_ (0)
   [junit4]   1>   /solr/configs (1)
   [junit4]   1>    /solr/configs/conf1 (11)
   [junit4]   1>     /solr/configs/conf1/solrconfig.snippet.randomindexconfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/open-exchange-rates.json (0)
   [junit4]   1>     DATA:
   [junit4]   1>         {
   [junit4]   1>           "disclaimer": "This data is not real, it was synthetically created to match currency.xml.  It is modeled after the data format available from openexchangerates.org.  See https://openexchangerates.org/documentation for details",
   [junit4]   1>           "license": "http://www.apache.org/licenses/LICENSE-2.0",
   [junit4]   1>           "timestamp": 1332070464,
   [junit4]   1>         
   [junit4]   1>         
   [junit4]   1>           "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
   [junit4]   1>         
   [junit4]   1>         
   [junit4]   1>           "base": "USD",
   [junit4]   1>           "rates": {
   [junit4]   1>             "USD": 1,
   [junit4]   1>             "JPY": 81.29,
   [junit4]   1>             "EUR": 2.5,
   [junit4]   1>             "GBP": 0.5,
   [junit4]   1>             "MXN": 2.0
   [junit4]   1>           }
   [junit4]   1>         }
   [junit4]   1>         
   [junit4]   1>     /solr/configs/conf1/enumsConfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/currency.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/schema.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/stopwords.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/old_synonyms.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/solrconfig.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/protwords.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>     /solr/configs/conf1/synonyms.txt (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>  /zookeeper (1)
   [junit4]   1>  DATA:
   [junit4]   1>      
   [junit4]   1> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.method=testDistribSearch -Dtests.seed=AECDF85261C18C80 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=in_ID -Dtests.timezone=America/Bahia -Dtests.file.encoding=UTF-8
   [junit4] FAILURE  207s J1 | ChaosMonkeyNothingIsSafeTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There were expected update fails expected:<0> but was:<1>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([AECDF85261C18C80:2F2B764A169EECBC]:0)
   [junit4]    > 	at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.doTest(ChaosMonkeyNothingIsSafeTest.java:202)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:843)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 904210 T748 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 206608 T747 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=in_ID, timezone=America/Bahia
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=1,free=287871472,total=414187520
   [junit4]   2> NOTE: All tests run in this JVM: [TestCodecSupport, SpellCheckComponentTest, TestAddFieldRealTimeGet, RAMDirectoryFactoryTest, ExpressionTest, TestCollapseQParserPlugin, TestStressRecovery, CurrencyFieldXmlFileTest, ZkNodePropsTest, TimeZoneUtilsTest, TestRandomMergePolicy, SignatureUpdateProcessorFactoryTest, OutputWriterTest, TestIndexingPerformance, AbstractAnalyticsStatsTest, TestDefaultSimilarityFactory, TestStressLucene, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestComponentsName, TestBinaryResponseWriter, SuggesterTSTTest, SuggesterWFSTTest, SortByFunctionTest, FieldFacetExtrasTest, TestFastLRUCache, TestHashPartitioner, TestSolrDeletionPolicy2, MigrateRouteKeyTest, ConvertedLegacyTest, TestStressReorder, TestSerializedLuceneMatchVersion, TestManagedSchemaFieldResource, BasicZkTest, TestFieldTypeResource, TestMultiCoreConfBootstrap, SpellCheckCollatorTest, SimplePostToolTest, TestFieldResource, TestDynamicFieldResource, TestDistributedSearch, ChaosMonkeyNothingIsSafeTest]
   [junit4] Completed on J1 in 206.90s, 1 test, 1 failure <<< FAILURES!

[...truncated 727 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:420: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:400: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:491: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1275: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:908: There were test failures: 357 suites, 1570 tests, 1 failure, 34 ignored (6 assumptions)

Total time: 74 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message