lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3880 - Still Failing
Date Fri, 12 Apr 2013 17:33:42 GMT
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3880/

1 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch

Error Message:
Wrong doc count on shard1_1 expected:<49> but was:<50>

Stack Trace:
java.lang.AssertionError: Wrong doc count on shard1_1 expected:<49> but was:<50>
	at __randomizedtesting.SeedInfo.seed([F1985D3A3B63351B:707ED3224C3C5527]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:274)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9134 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:05.913; org.apache.solr.BaseDistributedSearchTestCase; Setting hostContext system property: /ullu/h
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:05.918; org.apache.solr.SolrTestCaseJ4; ###Starting testDistribSearch
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:05.919; org.apache.solr.cloud.ZkTestServer; STARTING ZK TEST SERVER
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:05.920; org.apache.solr.cloud.ZkTestServer$ZKServerMain; Starting server
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.021; org.apache.solr.cloud.ZkTestServer; start zk server on port:37462
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.022; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.025; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@692796fe name:ZooKeeperConnection Watcher:127.0.0.1:37462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.026; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.026; org.apache.solr.common.cloud.SolrZkClient; makePath: /solr
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.042; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.044; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@72f241c name:ZooKeeperConnection Watcher:127.0.0.1:37462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.044; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.045; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.055; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/shards
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.057; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.059; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/shards
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.068; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.069; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.081; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.082; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/schema.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.185; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.186; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.189; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.189; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/protwords.txt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.204; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.204; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.207; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.208; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.211; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.212; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.215; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.216; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.219; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.219; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.430; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.435; org.eclipse.jetty.server.AbstractConnector; Started SocketConnector@127.0.0.1:37465
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.436; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.436; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.437; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365786846223
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.438; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365786846223/solr.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.438; org.apache.solr.core.CoreContainer; New CoreContainer 76012868
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.439; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365786846223/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.439; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365786846223/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.507; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.508; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.508; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.509; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.509; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.510; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.510; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.511; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.512; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.512; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.536; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.556; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:37462/solr
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.556; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.557; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.560; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@3a626c6e name:ZooKeeperConnection Watcher:127.0.0.1:37462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.561; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.579; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.593; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.595; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@10a9ad57 name:ZooKeeperConnection Watcher:127.0.0.1:37462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.596; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.603; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/queue
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.607; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/collection-queue-work
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.617; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.619; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37465_ullu%2Fh
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.621; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:37465_ullu%2Fh
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.630; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/election
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.642; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/leader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.652; org.apache.solr.cloud.Overseer; Overseer (id=89508206734540803-127.0.0.1:37465_ullu%2Fh-n_0000000000) starting
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.661; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/queue-work
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.673; org.apache.solr.cloud.OverseerCollectionProcessor; Process current queue of collection creations
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.673; org.apache.solr.common.cloud.SolrZkClient; makePath: /clusterstate.json
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.675; org.apache.solr.common.cloud.SolrZkClient; makePath: /aliases.json
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.677; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.680; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Starting to work on the main queue
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.684; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365786846223/collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.684; org.apache.solr.cloud.ZkController; Check for collection zkNode:control_collection
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.685; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.685; org.apache.solr.cloud.ZkController; Load collection config from:/collections/control_collection
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.686; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365786846223/collection1/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.688; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365786846223/collection1/lib/README' to classloader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.689; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365786846223/collection1/lib/classes/' to classloader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.753; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.814; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.915; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:06.922; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.537; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.550; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.554; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.569; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.574; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.579; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:07.580; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:07.580; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.580; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:07.581; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:07.581; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.582; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.617; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365786846223/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/control/data/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.617; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e12b144
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.617; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.618; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/control/data
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.619; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/control/data/index/
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:07.619; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.620; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/control/data/index
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.625; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@22c09510; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.625; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.627; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.628; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.628; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.629; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.629; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.629; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.630; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.630; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.630; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.639; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.644; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@12ae461e main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.645; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.645; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.648; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@12ae461e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.650; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:07.650; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.183; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.184; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":null,
[junit4:junit4]   1>   "numShards":"1",
[junit4:junit4]   1>   "shard":null,
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"down",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"control_collection",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37465_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37465/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.185; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection control_collection with numShards 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.185; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.221; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.653; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.653; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:37465/ullu/h collection:control_collection shard:shard1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.654; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.681; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.692; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.693; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.693; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:37465/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.693; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.694; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:37465/ullu/h/collection1/ has no replicas
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.694; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:37465/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:08.694; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.725; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.751; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.778; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:37465/ullu/h/collection1/ and leader is http://127.0.0.1:37465/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.778; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37465/ullu/h
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.778; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.778; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.779; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.780; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.782; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.782; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.783; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.799; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.800; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.802; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@2122e5cf name:ZooKeeperConnection Watcher:127.0.0.1:37462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.803; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.804; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:09.806; org.apache.solr.cloud.ChaosMonkey; monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.016; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.020; org.eclipse.jetty.server.AbstractConnector; Started SocketConnector@127.0.0.1:37469
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.021; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.021; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.022; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365786849806
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.022; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365786849806/solr.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.023; org.apache.solr.core.CoreContainer; New CoreContainer 945839218
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.024; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365786849806/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.024; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365786849806/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.092; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.093; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.093; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.094; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.094; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.095; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.095; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.096; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.096; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.097; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.119; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.138; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:37462/solr
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.139; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.140; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.143; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@49381978 name:ZooKeeperConnection Watcher:127.0.0.1:37462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.144; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.159; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.171; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.173; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@cd2c5ee name:ZooKeeperConnection Watcher:127.0.0.1:37462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.173; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:10.184; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.187; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37469_ullu%2Fh
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.189; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:37469_ullu%2Fh
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.192; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.192; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.192; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.192; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.210; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365786849806/collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.210; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.211; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.211; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.212; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365786849806/collection1/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.214; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365786849806/collection1/lib/README' to classloader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.215; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365786849806/collection1/lib/classes/' to classloader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.256; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.257; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"127.0.0.1:37465_ullu%2Fh_collection1",
[junit4:junit4]   1>   "numShards":"1",
[junit4:junit4]   1>   "shard":"shard1",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"active",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"control_collection",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37465_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37465/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.266; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.270; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.270; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.270; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.327; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.428; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:11.435; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.069; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.082; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.086; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.102; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.107; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.111; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:12.112; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:12.113; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.114; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:12.115; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:12.115; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.115; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.116; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365786849806/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.116; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e12b144
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.117; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.118; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.118; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1/index/
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:12.119; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.120; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1/index
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.126; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@677df38f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.126; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.130; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.130; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.131; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.132; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.132; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.133; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.133; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.134; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.134; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.146; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.154; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@6eecc6dc main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.155; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.155; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.160; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@6eecc6dc main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.163; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.163; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.774; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.775; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":null,
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":null,
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"down",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37469_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37469/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.775; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection collection1 with numShards 2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.775; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.780; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.780; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:12.780; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.166; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.166; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:37469/ullu/h collection:collection1 shard:shard1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.167; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.265; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.308; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.308; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.308; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:37469/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.309; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.309; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:37469/ullu/h/collection1/ has no replicas
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.309; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:37469/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:13.309; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.298; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.342; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.342; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.342; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.362; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:37469/ullu/h/collection1/ and leader is http://127.0.0.1:37469/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.362; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37469/ullu/h
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.362; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.362; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.363; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.365; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.366; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.367; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.367; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.590; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.593; org.eclipse.jetty.server.AbstractConnector; Started SocketConnector@127.0.0.1:37473
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.594; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.595; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.595; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365786854383
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.596; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365786854383/solr.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.596; org.apache.solr.core.CoreContainer; New CoreContainer 850099100
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.597; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365786854383/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.598; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365786854383/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.665; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.666; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.666; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.667; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.667; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.668; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.668; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.669; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.669; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.670; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.691; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.710; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:37462/solr
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.711; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.712; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.715; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@633c4226 name:ZooKeeperConnection Watcher:127.0.0.1:37462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.716; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.718; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.730; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.732; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@4e606c8b name:ZooKeeperConnection Watcher:127.0.0.1:37462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.733; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:14.745; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.748; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37473_ullu%2Fh
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.759; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:37473_ullu%2Fh
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.761; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.761; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.761; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.761; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.762; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.762; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.779; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.785; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365786854383/collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.785; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.786; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.786; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.787; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365786854383/collection1/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.789; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365786854383/collection1/lib/classes/' to classloader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.789; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365786854383/collection1/lib/README' to classloader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.840; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.856; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.857; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"127.0.0.1:37469_ullu%2Fh_collection1",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard1",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"active",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37469_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37469/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.870; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.870; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.870; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.870; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:15.901; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.002; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.009; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.633; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.642; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.644; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.657; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.661; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.664; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:16.665; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:16.665; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.666; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:16.666; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:16.667; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.667; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.667; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365786854383/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.667; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e12b144
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.668; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.669; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.669; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2/index/
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:16.670; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.670; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2/index
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.676; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@9a6bfa1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.676; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.679; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.679; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.679; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.680; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.680; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.681; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.681; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.681; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.682; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.690; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.696; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@2246b55a main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.696; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.697; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.701; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@2246b55a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.703; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:16.703; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.374; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.374; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":null,
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":null,
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"down",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37473_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37473/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.375; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.375; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.389; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.389; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.389; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.389; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.706; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.706; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:37473/ullu/h collection:collection1 shard:shard2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.707; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.724; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.727; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.727; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.727; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:37473/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.728; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.728; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:37473/ullu/h/collection1/ has no replicas
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.728; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:37473/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:17.729; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.893; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.944; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.944; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.944; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.944; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.958; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:37473/ullu/h/collection1/ and leader is http://127.0.0.1:37473/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.958; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37473/ullu/h
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.958; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.958; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.959; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.960; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.961; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.962; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:18.963; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.185; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.188; org.eclipse.jetty.server.AbstractConnector; Started SocketConnector@127.0.0.1:37478
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.189; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.190; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.190; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365786858978
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.191; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365786858978/solr.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.191; org.apache.solr.core.CoreContainer; New CoreContainer 1699050422
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.192; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365786858978/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.193; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365786858978/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.260; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.260; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.261; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.261; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.262; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.262; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.263; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.263; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.264; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.265; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.288; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.307; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:37462/solr
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.308; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.309; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.312; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1791125 name:ZooKeeperConnection Watcher:127.0.0.1:37462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.313; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.325; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.339; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.341; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@50cec118 name:ZooKeeperConnection Watcher:127.0.0.1:37462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.341; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:19.350; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.354; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37478_ullu%2Fh
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.356; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:37478_ullu%2Fh
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.358; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.358; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.359; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.359; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.359; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.360; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.360; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.374; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.375; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.380; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365786858978/collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.380; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.381; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.381; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.382; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365786858978/collection1/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.384; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365786858978/collection1/lib/README' to classloader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.385; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365786858978/collection1/lib/classes/' to classloader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.435; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.456; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.457; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"127.0.0.1:37473_ullu%2Fh_collection1",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard2",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"active",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37473_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37473/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.470; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.470; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.471; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.470; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.470; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.496; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.597; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:20.604; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.196; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.205; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.208; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.220; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.224; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.228; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:21.228; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:21.229; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.229; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:21.230; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:21.231; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.231; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.231; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365786858978/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.231; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e12b144
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.232; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.233; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.233; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index/
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:21.234; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.234; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.239; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@71f57f89; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.239; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.242; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.242; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.243; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.243; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.244; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.244; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.244; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.245; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.245; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.254; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.259; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@33f4a5ca main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.260; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.260; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.264; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@33f4a5ca main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.266; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.266; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.975; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.976; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":null,
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":null,
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"down",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37478_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37478/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.976; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.976; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.980; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.980; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.980; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.980; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:21.980; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.269; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.269; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:37478/ullu/h collection:collection1 shard:shard1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.275; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:37478/ullu/h/collection1/ and leader is http://127.0.0.1:37469/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.276; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37478/ullu/h
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.276; org.apache.solr.cloud.ZkController; Core needs to recover:collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.276; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.277; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.277; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.278; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.278; org.apache.solr.cloud.ZkController; publishing core=collection1 state=recovering
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.278; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.279; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.279; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.280; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.372; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.385; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:37478_ullu%2Fh_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.510; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.513; org.eclipse.jetty.server.AbstractConnector; Started SocketConnector@127.0.0.1:37482
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.514; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.514; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.515; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365786862295
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.515; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365786862295/solr.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.516; org.apache.solr.core.CoreContainer; New CoreContainer 1539521282
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.517; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365786862295/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.517; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365786862295/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.584; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.585; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.585; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.586; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.586; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.587; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.587; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.588; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.589; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.589; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.610; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.630; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:37462/solr
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.631; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.632; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.635; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@176b562b name:ZooKeeperConnection Watcher:127.0.0.1:37462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.636; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.648; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.660; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.662; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@4e91eae2 name:ZooKeeperConnection Watcher:127.0.0.1:37462/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.662; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:22.673; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.498; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.499; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"127.0.0.1:37478_ullu%2Fh_collection1",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard1",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"recovering",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37478_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37478/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.524; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.524; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.524; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.524; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.524; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.524; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.676; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37482_ullu%2Fh
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.685; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:37482_ullu%2Fh
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.687; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.687; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.687; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.687; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.687; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.687; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.688; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.688; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.698; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.699; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.699; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.699; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.704; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365786862295/collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.704; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.705; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.705; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.707; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365786862295/collection1/'
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.708; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365786862295/collection1/lib/classes/' to classloader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.709; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365786862295/collection1/lib/README' to classloader
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.759; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.821; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.922; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:23.929; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.388; org.apache.solr.handler.admin.CoreAdminHandler; Waited coreNodeName: 127.0.0.1:37478_ullu%2Fh_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.388; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:37478_ullu%252Fh_collection1&state=recovering&nodeName=127.0.0.1:37478_ullu%252Fh&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.564; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.577; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.581; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.596; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.602; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.607; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:24.608; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:24.609; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.609; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:24.610; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:24.610; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.611; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.611; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365786862295/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.611; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e12b144
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.612; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.613; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.614; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index/
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:24.615; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.615; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.624; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@23cc6983; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.625; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.628; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.629; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.629; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.630; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.631; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.631; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.632; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.633; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.633; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.648; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.656; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@3fa5e1cc main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.657; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.657; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.663; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@3fa5e1cc main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.666; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:24.666; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.028; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.029; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":null,
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":null,
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"down",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37482_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37482/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.029; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.029; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.033; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.033; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.033; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.033; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.033; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.033; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.669; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.669; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:37482/ullu/h collection:collection1 shard:shard2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.698; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:37482/ullu/h/collection1/ and leader is http://127.0.0.1:37473/ullu/h/collection1/
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.698; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37482/ullu/h
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.698; org.apache.solr.cloud.ZkController; Core needs to recover:collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.699; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.699; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.699; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.700; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.701; org.apache.solr.cloud.ZkController; publishing core=collection1 state=recovering
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.701; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.701; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.702; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.703; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.719; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.719; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.721; org.apache.solr.cloud.AbstractFullDistribZkTestBase; Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.721; org.apache.solr.cloud.AbstractDistribZkTestBase; Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.722; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:25.732; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:37482_ullu%2Fh_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.391; org.apache.solr.cloud.RecoveryStrategy; Attempting to PeerSync from http://127.0.0.1:37469/ullu/h/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.391; org.apache.solr.update.PeerSync; PeerSync: core=collection1 url=http://127.0.0.1:37478/ullu/h START replicas=[http://127.0.0.1:37469/ullu/h/collection1/] nUpdates=100
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:26.392; org.apache.solr.update.PeerSync; no frame of reference to tell of we've missed updates
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.393; org.apache.solr.cloud.RecoveryStrategy; PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.393; org.apache.solr.cloud.RecoveryStrategy; Starting Replication Recovery. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.393; org.apache.solr.cloud.RecoveryStrategy; Begin buffering updates. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.393; org.apache.solr.update.UpdateLog; Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.394; org.apache.solr.cloud.RecoveryStrategy; Attempting to replicate from http://127.0.0.1:37469/ullu/h/collection1/. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.394; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.400; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.407; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.412; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@677df38f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.413; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.417; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@677df38f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@677df38f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.418; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.419; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@41f07a7e realtime
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.420; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.420; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.421; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for  10000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.421; org.apache.solr.handler.SnapPuller;  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.423; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for  10000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.424; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.425; org.apache.solr.handler.SnapPuller; Master's generation: 2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.425; org.apache.solr.handler.SnapPuller; Slave's generation: 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.425; org.apache.solr.handler.SnapPuller; Starting replication process
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.429; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.430; org.apache.solr.handler.SnapPuller; Number of files in latest index in master: 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.431; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index.20130413041426430
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.432; org.apache.solr.handler.SnapPuller; Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index.20130413041426430 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2ed20a28; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.436; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.438; org.apache.solr.handler.SnapPuller; Total time taken for download : 0 secs
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.440; org.apache.solr.update.DefaultSolrCoreState; Creating new IndexWriter...
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.441; org.apache.solr.update.DefaultSolrCoreState; Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.445; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@71f57f89; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@71f57f89; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.446; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.446; org.apache.solr.update.DefaultSolrCoreState; New IndexWriter is ready to be used.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.447; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@1685853d main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.449; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@1685853d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.449; org.apache.solr.core.CachingDirectoryFactory; looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index.20130413041426430 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index.20130413041426430;done=true>>]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.450; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index.20130413041426430
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.450; org.apache.solr.core.CachingDirectoryFactory; Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index.20130413041426430
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.451; org.apache.solr.cloud.RecoveryStrategy; No replay needed. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.451; org.apache.solr.cloud.RecoveryStrategy; Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.452; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.452; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.453; org.apache.solr.cloud.RecoveryStrategy; Finished recovery process. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.550; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.551; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"127.0.0.1:37482_ullu%2Fh_collection1",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard2",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"recovering",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37482_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37482/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.554; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"127.0.0.1:37478_ullu%2Fh_collection1",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard1",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"active",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37478_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37478/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.567; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.567; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.567; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.567; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.567; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.567; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.724; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.734; org.apache.solr.handler.admin.CoreAdminHandler; Waited coreNodeName: 127.0.0.1:37482_ullu%2Fh_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:26.734; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:37482_ullu%252Fh_collection1&state=recovering&nodeName=127.0.0.1:37482_ullu%252Fh&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:27.726; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.728; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.736; org.apache.solr.cloud.RecoveryStrategy; Attempting to PeerSync from http://127.0.0.1:37473/ullu/h/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.736; org.apache.solr.update.PeerSync; PeerSync: core=collection1 url=http://127.0.0.1:37482/ullu/h START replicas=[http://127.0.0.1:37473/ullu/h/collection1/] nUpdates=100
[junit4:junit4]   1> WARN  - 2013-04-12 17:14:28.737; org.apache.solr.update.PeerSync; no frame of reference to tell of we've missed updates
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.737; org.apache.solr.cloud.RecoveryStrategy; PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.737; org.apache.solr.cloud.RecoveryStrategy; Starting Replication Recovery. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.737; org.apache.solr.cloud.RecoveryStrategy; Begin buffering updates. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.738; org.apache.solr.update.UpdateLog; Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.738; org.apache.solr.cloud.RecoveryStrategy; Attempting to replicate from http://127.0.0.1:37473/ullu/h/collection1/. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.738; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.748; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.756; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.761; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@9a6bfa1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.762; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.795; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@9a6bfa1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@9a6bfa1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.795; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.797; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@deab85e realtime
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.797; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.798; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 42
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.799; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for  10000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.799; org.apache.solr.handler.SnapPuller;  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.801; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for  10000
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.801; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.802; org.apache.solr.handler.SnapPuller; Master's generation: 2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.803; org.apache.solr.handler.SnapPuller; Slave's generation: 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.803; org.apache.solr.handler.SnapPuller; Starting replication process
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.806; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.807; org.apache.solr.handler.SnapPuller; Number of files in latest index in master: 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.808; org.apache.solr.core.CachingDirectoryFactory; return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index.20130413041428807
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.809; org.apache.solr.handler.SnapPuller; Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index.20130413041428807 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5af5a736; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.813; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.815; org.apache.solr.handler.SnapPuller; Total time taken for download : 0 secs
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.817; org.apache.solr.update.DefaultSolrCoreState; Creating new IndexWriter...
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.817; org.apache.solr.update.DefaultSolrCoreState; Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.822; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@23cc6983; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@23cc6983; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.823; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.823; org.apache.solr.update.DefaultSolrCoreState; New IndexWriter is ready to be used.
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.824; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@551258b2 main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.825; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@551258b2 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.826; org.apache.solr.core.CachingDirectoryFactory; looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index.20130413041428807 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index.20130413041428807;done=true>>]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.826; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index.20130413041428807
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.827; org.apache.solr.core.CachingDirectoryFactory; Removing directory before core close: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index.20130413041428807
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.828; org.apache.solr.cloud.RecoveryStrategy; No replay needed. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.828; org.apache.solr.cloud.RecoveryStrategy; Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.828; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.829; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:28.830; org.apache.solr.cloud.RecoveryStrategy; Finished recovery process. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.582; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.583; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"127.0.0.1:37482_ullu%2Fh_collection1",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard2",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"active",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37482_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37482/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.622; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.622; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.622; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.622; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.622; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.622; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.730; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.731; org.apache.solr.cloud.AbstractDistribZkTestBase; Recoveries finished - collection: collection1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.741; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.747; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@22c09510; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.748; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.754; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@22c09510; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@22c09510; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.754; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.756; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@58d5d86b main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.757; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.758; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@58d5d86b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.758; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.773; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.786; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@9a6bfa1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@9a6bfa1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.787; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.788; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@52bed5ea main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.789; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.790; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@52bed5ea main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.790; org.apache.solr.update.SolrCmdDistributor; Distrib commit to:[StdNode: http://127.0.0.1:37469/ullu/h/collection1/, StdNode: http://127.0.0.1:37478/ullu/h/collection1/, StdNode: http://127.0.0.1:37482/ullu/h/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.795; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.801; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@677df38f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@677df38f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.801; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.803; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@5ac47e23 main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.804; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.805; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@5ac47e23 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.805; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.809; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.809; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.823; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@71f57f89; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@71f57f89; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.824; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@23cc6983; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> 	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@23cc6983; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.824; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.825; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.828; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@3eb3e0a0 main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.828; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@163a244f main
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.828; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.829; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.829; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@3eb3e0a0 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.830; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@163a244f main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.830; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 21
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.831; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 22
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.832; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 59
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.833; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.836; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.838; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.841; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:29.843; org.apache.solr.core.SolrCore; [collection1] webapp=/ullu/h path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:31.845; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:31.848; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@df84ca8 name:ZooKeeperConnection Watcher:127.0.0.1:37462 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:31.849; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> / (2)
[junit4:junit4]   1> DATA:
[junit4:junit4]   1>     
[junit4:junit4]   1>  /solr (7)
[junit4:junit4]   1>   /solr/configs (1)
[junit4:junit4]   1>    /solr/configs/conf1 (9)
[junit4:junit4]   1>     /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
[junit4:junit4]   1>     DATA: ...supressed...
[junit4:junit4]   1>     /solr/configs/conf1/currency.xml (0)
[junit4:junit4]   1>     DATA: ...supressed...
[junit4:junit4]   1>     /solr/configs/conf1/protwords.txt (0)
[junit4:junit4]   1>     DATA: ...supressed...
[junit4:junit4]   1>     /solr/configs/conf1/synonyms.txt (0)
[junit4:junit4]   1>     DATA: ...supressed...
[junit4:junit4]   1>     /solr/configs/conf1/old_synonyms.txt (0)
[junit4:junit4]   1>     DATA: ...supressed...
[junit4:junit4]   1>     /solr/configs/conf1/solrconfig.xml (0)
[junit4:junit4]   1>     DATA: ...supressed...
[junit4:junit4]   1>     /solr/configs/conf1/stopwords.txt (0)
[junit4:junit4]   1>     DATA: ...supressed...
[junit4:junit4]   1>     /solr/configs/conf1/open-exchange-rates.json (0)
[junit4:junit4]   1>     DATA:
[junit4:junit4]   1>         {
[junit4:junit4]   1>           "disclaimer": "This data is not real, it was synthetically created to match currency.xml.  It is modeled after the data format available from openexchangerates.org.  See https://openexchangerates.org/documentation for details",
[junit4:junit4]   1>           "license": "http://www.apache.org/licenses/LICENSE-2.0",
[junit4:junit4]   1>           "timestamp": 1332070464,
[junit4:junit4]   1>         
[junit4:junit4]   1>         
[junit4:junit4]   1>           "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
[junit4:junit4]   1>         
[junit4:junit4]   1>         
[junit4:junit4]   1>           "base": "USD",
[junit4:junit4]   1>           "rates": {
[junit4:junit4]   1>             "USD": 1,
[junit4:junit4]   1>             "JPY": 81.29,
[junit4:junit4]   1>             "EUR": 2.5,
[junit4:junit4]   1>             "GBP": 0.5,
[junit4:junit4]   1>             "MXN": 2.0
[junit4:junit4]   1>           }
[junit4:junit4]   1>         }
[junit4:junit4]   1>         
[junit4:junit4]   1>     /solr/configs/conf1/schema.xml (0)
[junit4:junit4]   1>     DATA: ...supressed...
[junit4:junit4]   1>   /solr/clusterstate.json (0)
[junit4:junit4]   1>   DATA:
[junit4:junit4]   1>       {
[junit4:junit4]   1>         "collection1":{
[junit4:junit4]   1>           "shards":{
[junit4:junit4]   1>             "shard1":{
[junit4:junit4]   1>               "range":"80000000-ffffffff",
[junit4:junit4]   1>               "state":"active",
[junit4:junit4]   1>               "replicas":{
[junit4:junit4]   1>                 "127.0.0.1:37469_ullu%2Fh_collection1":{
[junit4:junit4]   1>                   "shard":"shard1",
[junit4:junit4]   1>                   "state":"active",
[junit4:junit4]   1>                   "core":"collection1",
[junit4:junit4]   1>                   "collection":"collection1",
[junit4:junit4]   1>                   "node_name":"127.0.0.1:37469_ullu%2Fh",
[junit4:junit4]   1>                   "base_url":"http://127.0.0.1:37469/ullu/h",
[junit4:junit4]   1>                   "leader":"true"},
[junit4:junit4]   1>                 "127.0.0.1:37478_ullu%2Fh_collection1":{
[junit4:junit4]   1>                   "shard":"shard1",
[junit4:junit4]   1>                   "state":"active",
[junit4:junit4]   1>                   "core":"collection1",
[junit4:junit4]   1>                   "collection":"collection1",
[junit4:junit4]   1>                   "node_name":"127.0.0.1:37478_ullu%2Fh",
[junit4:junit4]   1>                   "base_url":"http://127.0.0.1:37478/ullu/h"}}},
[junit4:junit4]   1>             "shard2":{
[junit4:junit4]   1>               "range":"0-7fffffff",
[junit4:junit4]   1>               "state":"active",
[junit4:junit4]   1>               "replicas":{
[junit4:junit4]   1>                 "127.0.0.1:37473_ullu%2Fh_collection1":{
[junit4:junit4]   1>                   "shard":"shard2",
[junit4:junit4]   1>                   "state":"active",
[junit4:junit4]   1>                   "core":"collection1",
[junit4:junit4]   1>                   "collection":"collection1",
[junit4:junit4]   1>                   "node_name":"127.0.0.1:37473_ullu%2Fh",
[junit4:junit4]   1>                   "base_url":"http://127.0.0.1:37473/ullu/h",
[junit4:junit4]   1>                   "leader":"true"},
[junit4:junit4]   1>                 "127.0.0.1:37482_ullu%2Fh_collection1":{
[junit4:junit4]   1>                   "shard":"shard2",
[junit4:junit4]   1>                   "state":"active",
[junit4:junit4]   1>                   "core":"collection1",
[junit4:junit4]   1>                   "collection":"collection1",
[junit4:junit4]   1>                   "node_name":"127.0.0.1:37482_ullu%2Fh",
[junit4:junit4]   1>                   "base_url":"http://127.0.0.1:37482/ullu/h"}}}},
[junit4:junit4]   1>           "router":"compositeId"},
[junit4:junit4]   1>         "control_collection":{
[junit4:junit4]   1>           "shards":{"shard1":{
[junit4:junit4]   1>               "range":"80000000-7fffffff",
[junit4:junit4]   1>               "state":"active",
[junit4:junit4]   1>               "replicas":{"127.0.0.1:37465_ullu%2Fh_collection1":{
[junit4:junit4]   1>                   "shard":"shard1",
[junit4:junit4]   1>                   "state":"active",
[junit4:junit4]   1>                   "core":"collection1",
[junit4:junit4]   1>                   "collection":"control_collection",
[junit4:junit4]   1>                   "node_name":"127.0.0.1:37465_ullu%2Fh",
[junit4:junit4]   1>                   "base_url":"http://127.0.0.1:37465/ullu/h",
[junit4:junit4]   1>                   "leader":"true"}}}},
[junit4:junit4]   1>           "router":"compositeId"}}
[junit4:junit4]   1>   /solr/aliases.json (0)
[junit4:junit4]   1>   /solr/live_nodes (5)
[junit4:junit4]   1>    /solr/live_nodes/127.0.0.1:37482_ullu%2Fh (0)
[junit4:junit4]   1>    /solr/live_nodes/127.0.0.1:37478_ullu%2Fh (0)
[junit4:junit4]   1>    /solr/live_nodes/127.0.0.1:37469_ullu%2Fh (0)
[junit4:junit4]   1>    /solr/live_nodes/127.0.0.1:37473_ullu%2Fh (0)
[junit4:junit4]   1>    /solr/live_nodes/127.0.0.1:37465_ullu%2Fh (0)
[junit4:junit4]   1>   /solr/overseer (3)
[junit4:junit4]   1>   DATA:
[junit4:junit4]   1>       
[junit4:junit4]   1>    /solr/overseer/queue (0)
[junit4:junit4]   1>    /solr/overseer/queue-work (0)
[junit4:junit4]   1>    /solr/overseer/collection-queue-work (0)
[junit4:junit4]   1>   /solr/collections (2)
[junit4:junit4]   1>    /solr/collections/collection1 (3)
[junit4:junit4]   1>    DATA:
[junit4:junit4]   1>        {"configName":"conf1"}
[junit4:junit4]   1>     /solr/collections/collection1/shards (0)
[junit4:junit4]   1>     /solr/collections/collection1/leader_elect (2)
[junit4:junit4]   1>      /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4]   1>       /solr/collections/collection1/leader_elect/shard1/election (2)
[junit4:junit4]   1>        /solr/collections/collection1/leader_elect/shard1/election/89508206734540806-127.0.0.1:37469_ullu%2Fh_collection1-n_0000000000 (0)
[junit4:junit4]   1>        /solr/collections/collection1/leader_elect/shard1/election/89508206734540810-127.0.0.1:37478_ullu%2Fh_collection1-n_0000000001 (0)
[junit4:junit4]   1>      /solr/collections/collection1/leader_elect/shard2 (1)
[junit4:junit4]   1>       /solr/collections/collection1/leader_elect/shard2/election (2)
[junit4:junit4]   1>        /solr/collections/collection1/leader_elect/shard2/election/89508206734540812-127.0.0.1:37482_ullu%2Fh_collection1-n_0000000001 (0)
[junit4:junit4]   1>        /solr/collections/collection1/leader_elect/shard2/election/89508206734540808-127.0.0.1:37473_ullu%2Fh_collection1-n_0000000000 (0)
[junit4:junit4]   1>     /solr/collections/collection1/leaders (2)
[junit4:junit4]   1>      /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"collection1",
[junit4:junit4]   1>            "node_name":"127.0.0.1:37469_ullu%2Fh",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:37469/ullu/h"}
[junit4:junit4]   1>      /solr/collections/collection1/leaders/shard2 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"collection1",
[junit4:junit4]   1>            "node_name":"127.0.0.1:37473_ullu%2Fh",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:37473/ullu/h"}
[junit4:junit4]   1>    /solr/collections/control_collection (3)
[junit4:junit4]   1>    DATA:
[junit4:junit4]   1>        {"configName":"conf1"}
[junit4:junit4]   1>     /solr/collections/control_collection/shards (0)
[junit4:junit4]   1>     /solr/collections/control_collection/leader_elect (1)
[junit4:junit4]   1>      /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4]   1>       /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4]   1>        /solr/collections/control_collection/leader_elect/shard1/election/89508206734540803-127.0.0.1:37465_ullu%2Fh_collection1-n_0000000000 (0)
[junit4:junit4]   1>     /solr/collections/control_collection/leaders (1)
[junit4:junit4]   1>      /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4]   1>      DATA:
[junit4:junit4]   1>          {
[junit4:junit4]   1>            "core":"collection1",
[junit4:junit4]   1>            "node_name":"127.0.0.1:37465_ullu%2Fh",
[junit4:junit4]   1>            "base_url":"http://127.0.0.1:37465/ullu/h"}
[junit4:junit4]   1>   /solr/overseer_elect (2)
[junit4:junit4]   1>    /solr/overseer_elect/election (5)
[junit4:junit4]   1>     /solr/overseer_elect/election/89508206734540806-127.0.0.1:37469_ullu%2Fh-n_0000000001 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89508206734540803-127.0.0.1:37465_ullu%2Fh-n_0000000000 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89508206734540812-127.0.0.1:37482_ullu%2Fh-n_0000000004 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89508206734540810-127.0.0.1:37478_ullu%2Fh-n_0000000003 (0)
[junit4:junit4]   1>     /solr/overseer_elect/election/89508206734540808-127.0.0.1:37473_ullu%2Fh-n_0000000002 (0)
[junit4:junit4]   1>    /solr/overseer_elect/leader (0)
[junit4:junit4]   1>    DATA:
[junit4:junit4]   1>        {"id":"89508206734540803-127.0.0.1:37465_ullu%2Fh-n_0000000000"}
[junit4:junit4]   1>  /zookeeper (1)
[junit4:junit4]   1>  DATA:
[junit4:junit4]   1>      
[junit4:junit4]   1> 
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.026; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={update.distrib=FROMLEADER&_version_=-1432131335113146368&update.from=http://127.0.0.1:37469/ullu/h/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1432131335113146368)} 0 2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.030; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={update.distrib=FROMLEADER&_version_=-1432131335119437824&update.from=http://127.0.0.1:37473/ullu/h/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1432131335119437824)} 0 2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.031; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1432131335119437824)} 0 7
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.032; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1432131335113146368)} 0 15
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.038; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[0 (1432131335132020736)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.047; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37469/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1432131335136215040)]} 0 2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.048; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[0 (1432131335136215040)]} 0 8
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.052; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[1 (1432131335147749376)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.059; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37469/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1432131335151943680)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.060; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[1 (1432131335151943680)]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.064; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[2 (1432131335160332288)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.075; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37473/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1432131335164526592)]} 0 2
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.076; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[2 (1432131335164526592)]} 0 9
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.080; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[3 (1432131335177109504)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.087; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37473/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1432131335181303808)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.088; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[3 (1432131335181303808)]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.092; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[4 (1432131335189692416)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.099; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37469/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1432131335193886720)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.100; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[4 (1432131335193886720)]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.104; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[5 (1432131335202275328)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.115; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37473/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1432131335209615360)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.116; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37469/ullu/h/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1432131335209615360)]} 0 6
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.117; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[5]} 0 10
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.121; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[6 (1432131335220101120)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.131; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37473/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1432131335227441152)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.132; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37469/ullu/h/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1432131335227441152)]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.133; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[6]} 0 9
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.137; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[7 (1432131335236878336)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.144; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37473/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1432131335241072640)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.145; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[7 (1432131335241072640)]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.149; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[8 (1432131335249461248)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.160; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37469/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1432131335256801280)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.161; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37473/ullu/h/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1432131335256801280)]} 0 6
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.162; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[8]} 0 10
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.166; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[9 (1432131335267287040)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.176; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37473/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1432131335274627072)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.177; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37469/ullu/h/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1432131335274627072)]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.178; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[9]} 0 9
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.182; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[10 (1432131335284064256)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.192; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37469/ullu/h/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1432131335291404288)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.193; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={distrib.from=http://127.0.0.1:37473/ullu/h/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1432131335291404288)]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.194; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[10]} 0 9
[junit4:junit4]   1> INFO  - 2013-04-12 17:14:32.198; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/ullu/h path=/update params={wt=javabin&version=2} {add=[11 (1432

[...truncated too long message...]

ull,
[junit4:junit4]   1>   "core":"collection1_shard1_0_replica1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:37478_ullu%2Fh",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:37478/ullu/h"}
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:04.038; org.apache.solr.cloud.ChaosMonkey; monkey: stop shard! 37482
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:04.038; org.apache.solr.core.CoreContainer; Shutting down CoreContainer instance=1539521282
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:04.039; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.048; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.048; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.055; org.apache.solr.core.SolrCore; [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3bbcb67
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.057; org.apache.solr.update.DirectUpdateHandler2; closing DirectUpdateHandler2{commits=4,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=98,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.058; org.apache.solr.update.SolrCoreState; Closing SolrCoreState
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.058; org.apache.solr.update.DefaultSolrCoreState; SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.058; org.apache.solr.update.DefaultSolrCoreState; closing IndexWriter with IndexWriterCloser
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.061; org.apache.solr.core.SolrCore; [collection1] Closing main searcher on request.
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.062; org.apache.solr.core.CachingDirectoryFactory; Closing NRTCachingDirectoryFactory - 2 directories currently being tracked
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.062; org.apache.solr.core.CachingDirectoryFactory; looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4;done=false>>]
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.063; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.063; org.apache.solr.core.CachingDirectoryFactory; looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index;done=false>>]
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.064; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365786845919/jetty4/index
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.065; org.apache.solr.cloud.Overseer$ClusterStateUpdater; According to ZK I (id=89508206734540812-127.0.0.1:37482_ullu%2Fh-n_0000000004) am no longer a leader.
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.066; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.066; org.eclipse.jetty.server.handler.ContextHandler; stopped o.e.j.s.ServletContextHandler{/ullu/h,null}
[junit4:junit4]   1> WARN  - 2013-04-12 17:16:05.066; org.apache.solr.common.cloud.ZkStateReader$2; ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   1> WARN  - 2013-04-12 17:16:05.067; org.apache.solr.common.cloud.ZkStateReader$3; ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.198; org.apache.solr.SolrTestCaseJ4; ###Ending testDistribSearch
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.201; org.apache.solr.cloud.ZkTestServer; connecting to 127.0.0.1:37462 37462
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeyShardSplitTest-1365786845919
[junit4:junit4]   2> check const of shard1
[junit4:junit4]   2> client0
[junit4:junit4]   2> PROPS:127.0.0.1:37469_ullu%2Fh_collection1:{"shard":"shard1","state":"down","core":"collection1","collection":"collection1","node_name":"127.0.0.1:37469_ullu%2Fh","base_url":"http://127.0.0.1:37469/ullu/h"}
[junit4:junit4]   2> error contacting client: IOException occured when talking to server at: http://127.0.0.1:37469/ullu/h/collection1
[junit4:junit4]   2> 
[junit4:junit4]   2> client1
[junit4:junit4]   2> PROPS:127.0.0.1:37478_ullu%2Fh_collection1:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:37478_ullu%2Fh","base_url":"http://127.0.0.1:37478/ullu/h"}
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:56
[junit4:junit4]   2> 
[junit4:junit4]   2> check const of shard2
[junit4:junit4]   2> client0
[junit4:junit4]   2> PROPS:127.0.0.1:37473_ullu%2Fh_collection1:{"shard":"shard2","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:37473_ullu%2Fh","base_url":"http://127.0.0.1:37473/ullu/h","leader":"true"}
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:44
[junit4:junit4]   2> 
[junit4:junit4]   2> client1
[junit4:junit4]   2> PROPS:127.0.0.1:37482_ullu%2Fh_collection1:{"shard":"shard2","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:37482_ullu%2Fh","base_url":"http://127.0.0.1:37482/ullu/h"}
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:44
[junit4:junit4]   2> 
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeyShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=F1985D3A3B63351B -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=no_NO -Dtests.timezone=Pacific/Ponape -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE  120s J0 | ChaosMonkeyShardSplitTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: Wrong doc count on shard1_1 expected:<49> but was:<50>
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([F1985D3A3B63351B:707ED3224C3C5527]:0)
[junit4:junit4]    > 	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:274)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   1> INFO  - 2013-04-12 17:16:05.708; org.apache.solr.SolrTestCaseJ4; ###deleteCore
[junit4:junit4]   2> 119799 T1132 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=no_NO, timezone=Pacific/Ponape
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_17 (64-bit)/cpus=16,threads=2,free=353772784,total=441450496
[junit4:junit4]   2> NOTE: All tests run in this JVM: [ShardSplitTest, TermsComponentTest, TestExtendedDismaxParser, OverseerTest, TestGroupingSearch, TestDocumentBuilder, DistributedTermsComponentTest, SuggesterTest, UnloadDistributedZkTest, LukeRequestHandlerTest, TestSolrCoreProperties, AnalysisAfterCoreReloadTest, TestSchemaResource, BinaryUpdateRequestHandlerTest, TestUniqueKeyFieldResource, TestFunctionQuery, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J0 in 120.33s, 1 test, 1 failure <<< FAILURES!

[...truncated 636 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:375: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:355: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:438: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1231: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:875: There were test failures: 289 suites, 1200 tests, 1 failure, 27 ignored (8 assumptions)

Total time: 59 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message