lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-Tests-4.x-Java6 - Build # 1511 - Still Failing
Date Mon, 15 Apr 2013 03:53:15 GMT
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java6/1511/

1 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch

Error Message:
Wrong doc count on shard1_1 expected:<322> but was:<323>

Stack Trace:
java.lang.AssertionError: Wrong doc count on shard1_1 expected:<322> but was:<323>
	at __randomizedtesting.SeedInfo.seed([32D28CFFA5436A68:B33402E7D21C0A54]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.cloud.ShardSplitTest.checkDocCountsAndShardStates(ShardSplitTest.java:155)
	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:154)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:679)




Build Log:
[...truncated 9174 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.072; org.apache.solr.BaseDistributedSearchTestCase; Setting hostContext system property: /jj/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.075; org.apache.solr.SolrTestCaseJ4; ###Starting testDistribSearch
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.076; org.apache.solr.cloud.ZkTestServer; STARTING ZK TEST SERVER
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.077; org.apache.solr.cloud.ZkTestServer$ZKServerMain; Starting server
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.178; org.apache.solr.cloud.ZkTestServer; start zk server on port:18952
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.179; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.182; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@71636e40 name:ZooKeeperConnection Watcher:127.0.0.1:18952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.182; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.183; org.apache.solr.common.cloud.SolrZkClient; makePath: /solr
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.197; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.221; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@149329ac name:ZooKeeperConnection Watcher:127.0.0.1:18952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.221; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.221; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.242; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/shards
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.245; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.257; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/shards
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.260; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.261; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.323; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.324; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/schema.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.428; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.429; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.446; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.447; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/protwords.txt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.450; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.450; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.458; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.459; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.467; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.469; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.494; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.495; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.499; org.apache.solr.cloud.AbstractZkTestCase; put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.499; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.745; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.753; org.eclipse.jetty.server.AbstractConnector; Started SocketConnector@127.0.0.1:18955
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.754; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.755; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.755; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365997526526
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.756; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365997526526/solr.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.765; org.apache.solr.core.CoreContainer; New CoreContainer 100689678
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.766; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365997526526/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.766; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365997526526/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.832; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.832; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.833; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.833; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.834; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.834; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.835; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.835; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.836; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.836; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.856; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.875; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:18952/solr
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.876; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.877; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.888; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@4438487f name:ZooKeeperConnection Watcher:127.0.0.1:18952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.888; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.891; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.901; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.905; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@2fb28819 name:ZooKeeperConnection Watcher:127.0.0.1:18952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.905; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.914; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/queue
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.925; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/collection-queue-work
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.935; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.937; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18955_jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.939; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:18955_jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:26.983; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/election
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.080; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/leader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.084; org.apache.solr.cloud.Overseer; Overseer (id=89522013869309955-127.0.0.1:18955_jj-n_0000000000) starting
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.114; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/queue-work
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.151; org.apache.solr.cloud.OverseerCollectionProcessor; Process current queue of collection creations
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.152; org.apache.solr.common.cloud.SolrZkClient; makePath: /clusterstate.json
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.201; org.apache.solr.common.cloud.SolrZkClient; makePath: /aliases.json
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.203; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.205; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Starting to work on the main queue
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.207; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365997526526/collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.208; org.apache.solr.cloud.ZkController; Check for collection zkNode:control_collection
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.208; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.209; org.apache.solr.cloud.ZkController; Load collection config from:/collections/control_collection
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.210; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365997526526/collection1/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.211; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365997526526/collection1/lib/README' to classloader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.212; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365997526526/collection1/lib/classes/' to classloader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.260; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.319; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.421; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:27.428; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.041; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.052; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.057; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.076; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.081; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.085; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:28.086; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:28.086; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.087; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:28.088; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:28.088; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.088; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.089; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365997526526/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/control/data/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.089; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c29d65b
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.089; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.090; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/control/data
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.090; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/control/data/index/
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:28.091; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.091; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/control/data/index
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.095; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@51cc7fbc lockFactory=org.apache.lucene.store.NativeFSLockFactory@495d56fd),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.095; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.113; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.113; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.114; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.114; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.115; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.115; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.115; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.116; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.116; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.123; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.129; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@18b46bac main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.130; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/control/data/tlog
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.131; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.131; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.134; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@18b46bac main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.136; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.136; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.708; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.709; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"1",
[junit4:junit4]   1>   "numShards":"1",
[junit4:junit4]   1>   "shard":null,
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"down",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"control_collection",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18955_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18955/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.709; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection control_collection with numShards 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.710; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:28.747; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.142; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.142; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:18955/jj collection:control_collection shard:shard1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.143; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.180; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.189; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.190; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.190; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:18955/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.190; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.190; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:18955/jj/collection1/ has no replicas
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.191; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:18955/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:29.191; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.313; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.335; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.382; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:18955/jj/collection1/ and leader is http://127.0.0.1:18955/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.382; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18955/jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.382; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.382; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.383; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.384; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.385; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.386; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.386; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.395; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.397; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.399; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@2408e9ff name:ZooKeeperConnection Watcher:127.0.0.1:18952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.399; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.401; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.403; org.apache.solr.cloud.ChaosMonkey; monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.536; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.538; org.eclipse.jetty.server.AbstractConnector; Started SocketConnector@127.0.0.1:18959
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.539; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.539; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.539; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365997530403
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.540; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365997530403/solr.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.540; org.apache.solr.core.CoreContainer; New CoreContainer 388934360
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.540; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365997530403/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.541; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365997530403/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.577; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.578; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.578; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.578; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.579; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.579; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.579; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.580; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.580; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.580; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.594; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.606; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:18952/solr
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.607; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.608; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.614; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@39406765 name:ZooKeeperConnection Watcher:127.0.0.1:18952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.614; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.715; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.723; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.725; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@67c19bf2 name:ZooKeeperConnection Watcher:127.0.0.1:18952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.726; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:30.813; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.816; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18959_jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.819; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:18959_jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.822; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.822; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.822; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.823; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.839; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365997530403/collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.839; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.840; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.840; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.840; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.841; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"1",
[junit4:junit4]   1>   "numShards":"1",
[junit4:junit4]   1>   "shard":"shard1",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"active",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"control_collection",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18955_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18955/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.841; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365997530403/collection1/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.843; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365997530403/collection1/lib/classes/' to classloader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.843; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365997530403/collection1/lib/README' to classloader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.872; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.872; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.872; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.899; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:31.962; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.063; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.069; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.675; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.686; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.690; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.702; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.707; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.711; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:32.712; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:32.712; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.713; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:32.713; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:32.714; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.714; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.714; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365997530403/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.715; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c29d65b
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.715; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.716; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.716; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty1/index/
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:32.716; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.717; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty1/index
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.722; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2506007a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d980059),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.723; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.739; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.739; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.740; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.740; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.741; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.741; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.742; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.742; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.743; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.750; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.757; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@73e5f8bc main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.757; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty1/tlog
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.758; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.758; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.762; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@73e5f8bc main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.764; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:32.764; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.412; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.413; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"2",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":null,
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"down",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18959_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18959/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.413; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection collection1 with numShards 2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.414; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.431; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.431; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.431; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.767; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.767; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:18959/jj collection:collection1 shard:shard1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.768; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.807; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.818; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.818; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.819; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:18959/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.819; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.819; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:18959/jj/collection1/ has no replicas
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.819; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:18959/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:33.819; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:34.939; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:34.985; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:34.985; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:34.985; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.020; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:18959/jj/collection1/ and leader is http://127.0.0.1:18959/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.020; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18959/jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.020; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.020; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.020; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.022; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.023; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.023; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.024; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.187; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.190; org.eclipse.jetty.server.AbstractConnector; Started SocketConnector@127.0.0.1:18962
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.190; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.191; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.191; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365997535033
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.191; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365997535033/solr.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.192; org.apache.solr.core.CoreContainer; New CoreContainer 673303715
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.192; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365997535033/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.192; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365997535033/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.228; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.229; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.229; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.229; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.230; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.230; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.230; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.231; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.231; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.231; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.244; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.256; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:18952/solr
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.257; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.258; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.261; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@35b7f804 name:ZooKeeperConnection Watcher:127.0.0.1:18952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.262; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.281; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.291; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.294; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@3e235306 name:ZooKeeperConnection Watcher:127.0.0.1:18952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.294; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:35.309; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.312; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18962_jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.316; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:18962_jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.324; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.325; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.325; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.325; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.326; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.326; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.332; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.335; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365997535033/collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.336; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.336; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.336; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.338; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365997535033/collection1/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.339; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365997535033/collection1/lib/README' to classloader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.339; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365997535033/collection1/lib/classes/' to classloader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.388; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.448; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.499; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.500; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"2",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard1",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"active",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18959_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18959/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.523; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.523; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.523; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.549; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.549; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:36.556; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.210; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.226; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.230; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.243; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.247; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.250; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:37.250; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:37.251; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.251; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:37.252; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:37.252; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.253; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.253; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365997535033/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty2/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.253; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c29d65b
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.253; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.254; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.254; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty2/index/
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:37.254; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.255; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty2/index
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.257; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4259d58d lockFactory=org.apache.lucene.store.NativeFSLockFactory@538e56b7),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.257; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.291; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.292; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.292; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.292; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.293; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.293; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.293; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.294; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.294; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.298; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.302; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@6e7ebdf3 main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.303; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty2/tlog
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.303; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.304; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.311; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@6e7ebdf3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.312; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:37.313; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.027; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.028; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"3",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":null,
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"down",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18962_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18962/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.029; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.029; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.078; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.078; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.078; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.078; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.357; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.357; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:18962/jj collection:collection1 shard:shard2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.358; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.400; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.418; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.418; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.419; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:18962/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.419; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.419; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:18962/jj/collection1/ has no replicas
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.420; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:18962/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:38.420; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.582; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.648; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.648; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.648; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.648; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.672; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:18962/jj/collection1/ and leader is http://127.0.0.1:18962/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.672; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18962/jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.672; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.672; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.673; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.675; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.676; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.676; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.677; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.841; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.844; org.eclipse.jetty.server.AbstractConnector; Started SocketConnector@127.0.0.1:18965
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.845; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.845; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.845; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365997539686
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.846; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365997539686/solr.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.846; org.apache.solr.core.CoreContainer; New CoreContainer 2092836234
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.847; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365997539686/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.847; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365997539686/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.884; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.884; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.885; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.885; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.885; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.886; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.886; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.886; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.887; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.887; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.900; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.912; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:18952/solr
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.913; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.914; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.916; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@154726f3 name:ZooKeeperConnection Watcher:127.0.0.1:18952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.917; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.941; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.947; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.949; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@62629553 name:ZooKeeperConnection Watcher:127.0.0.1:18952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.950; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:39.981; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.984; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18965_jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.986; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:18965_jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.989; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.989; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.990; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.990; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.990; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.990; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.991; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.991; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:40.998; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.003; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365997539686/collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.003; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.004; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.004; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.005; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365997539686/collection1/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.006; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365997539686/collection1/lib/README' to classloader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.007; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365997539686/collection1/lib/classes/' to classloader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.054; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.111; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.177; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.178; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"3",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard2",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"active",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18962_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18962/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.197; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.197; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.197; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.197; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.212; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.212; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.218; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.854; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.866; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.870; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.882; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.887; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.891; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:41.892; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:41.893; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.894; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:41.895; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:41.895; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.895; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.895; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365997539686/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.896; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c29d65b
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.896; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.897; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.897; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3/index/
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:41.898; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.898; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3/index
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.901; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6ba65089 lockFactory=org.apache.lucene.store.NativeFSLockFactory@415c85ad),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.901; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.925; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.925; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.926; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.926; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.927; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.927; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.927; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.928; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.928; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.935; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.941; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@282e4840 main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.941; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3/tlog
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.943; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.943; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.947; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@282e4840 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.949; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:41.949; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.701; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.702; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"4",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":null,
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"down",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18965_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18965/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.702; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.703; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.725; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.725; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.726; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.725; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.725; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.989; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.989; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:18965/jj collection:collection1 shard:shard1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.995; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:18965/jj/collection1/ and leader is http://127.0.0.1:18959/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.995; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18965/jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.995; org.apache.solr.cloud.ZkController; Core needs to recover:collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.996; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.996; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.996; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.997; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.997; org.apache.solr.cloud.ZkController; publishing core=collection1 state=recovering
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.998; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.998; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.999; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.999; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:42.999; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.014; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.232; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.236; org.eclipse.jetty.server.AbstractConnector; Started SocketConnector@127.0.0.1:18969
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.236; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.237; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.237; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365997543012
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.238; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365997543012/solr.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.238; org.apache.solr.core.CoreContainer; New CoreContainer 1169555301
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.239; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365997543012/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.239; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365997543012/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.294; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.295; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.295; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.296; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.296; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.297; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.297; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.298; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.298; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.299; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.318; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.336; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:18952/solr
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.337; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.338; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.346; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@33cbbc62 name:ZooKeeperConnection Watcher:127.0.0.1:18952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.347; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.376; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.385; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.387; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@7268f48 name:ZooKeeperConnection Watcher:127.0.0.1:18952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.388; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:43.414; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.229; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.230; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"4",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard1",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"recovering",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18965_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18965/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.280; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.280; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.281; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.280; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.280; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.280; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.417; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18969_jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.419; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:18969_jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.433; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.433; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.433; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.433; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.434; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.435; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.434; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.435; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.435; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.436; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.437; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.437; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.441; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365997543012/collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.442; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.442; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.442; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.444; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365997543012/collection1/'
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.445; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365997543012/collection1/lib/classes/' to classloader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.446; org.apache.solr.core.SolrResourceLoader; Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365997543012/collection1/lib/README' to classloader
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.498; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_43
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.554; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.655; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:44.661; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.016; org.apache.solr.handler.admin.CoreAdminHandler; Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.016; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:18965_jj&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.271; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.282; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.286; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.298; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.302; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.306; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:45.307; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:45.308; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.308; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:45.309; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:45.309; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.310; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.310; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365997543012/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.310; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c29d65b
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.311; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.311; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.312; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4/index/
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:45.312; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.312; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4/index
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.315; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16e3da96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1086cdc2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.315; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.324; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.325; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.325; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.326; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.326; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.327; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.327; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.328; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.328; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.335; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.341; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@b76b62f main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.341; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4/tlog
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.342; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.343; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.347; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@b76b62f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.348; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.349; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.790; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.791; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"5",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":null,
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"down",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18969_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18969/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.792; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.792; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.813; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.813; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.813; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.813; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.813; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:45.813; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.352; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.352; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:18969/jj collection:collection1 shard:shard2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.361; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:18969/jj/collection1/ and leader is http://127.0.0.1:18962/jj/collection1/
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.362; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18969/jj
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.362; org.apache.solr.cloud.ZkController; Core needs to recover:collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.362; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.363; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.364; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.364; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.364; org.apache.solr.cloud.ZkController; publishing core=collection1 state=recovering
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.365; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.365; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.366; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.367; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.417; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.417; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.419; org.apache.solr.cloud.AbstractFullDistribZkTestBase; Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.419; org.apache.solr.cloud.AbstractDistribZkTestBase; Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.420; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:46.428; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.018; org.apache.solr.cloud.RecoveryStrategy; Attempting to PeerSync from http://127.0.0.1:18959/jj/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.018; org.apache.solr.update.PeerSync; PeerSync: core=collection1 url=http://127.0.0.1:18965/jj START replicas=[http://127.0.0.1:18959/jj/collection1/] nUpdates=100
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:47.019; org.apache.solr.update.PeerSync; no frame of reference to tell of we've missed updates
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.020; org.apache.solr.cloud.RecoveryStrategy; PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.020; org.apache.solr.cloud.RecoveryStrategy; Starting Replication Recovery. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.020; org.apache.solr.cloud.RecoveryStrategy; Begin buffering updates. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.020; org.apache.solr.update.UpdateLog; Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.021; org.apache.solr.cloud.RecoveryStrategy; Attempting to replicate from http://127.0.0.1:18959/jj/collection1/. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.021; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.026; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.033; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.097; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2506007a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d980059),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.097; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.098; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2506007a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d980059),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2506007a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d980059),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.098; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.099; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@24edda22 realtime
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.099; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.100; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 67
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.101; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for  10000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.101; org.apache.solr.handler.SnapPuller;  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.103; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for  10000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.103; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.104; org.apache.solr.handler.SnapPuller; Master's generation: 2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.104; org.apache.solr.handler.SnapPuller; Slave's generation: 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.104; org.apache.solr.handler.SnapPuller; Starting replication process
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.107; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.107; org.apache.solr.handler.SnapPuller; Number of files in latest index in master: 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.109; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3/index.20130415104547108
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.109; org.apache.solr.handler.SnapPuller; Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@2815a96a lockFactory=org.apache.lucene.store.NativeFSLockFactory@13f9703) fullCopy=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.112; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.113; org.apache.solr.handler.SnapPuller; Total time taken for download : 0 secs
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.113; org.apache.solr.update.DefaultSolrCoreState; Creating new IndexWriter...
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.114; org.apache.solr.update.DefaultSolrCoreState; Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.135; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6ba65089 lockFactory=org.apache.lucene.store.NativeFSLockFactory@415c85ad),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6ba65089 lockFactory=org.apache.lucene.store.NativeFSLockFactory@415c85ad),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.136; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.136; org.apache.solr.update.DefaultSolrCoreState; New IndexWriter is ready to be used.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.136; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@41a1ee2a main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.137; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@41a1ee2a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.138; org.apache.solr.core.CachingDirectoryFactory; looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3/index.20130415104547108 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3/index.20130415104547108;done=true>>]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.138; org.apache.solr.core.CachingDirectoryFactory; Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3/index.20130415104547108
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.138; org.apache.solr.core.CachingDirectoryFactory; Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty3/index.20130415104547108
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.138; org.apache.solr.cloud.RecoveryStrategy; No replay needed. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.138; org.apache.solr.cloud.RecoveryStrategy; Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.138; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.139; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.169; org.apache.solr.cloud.RecoveryStrategy; Finished recovery process. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.318; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.319; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"5",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard2",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"recovering",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18969_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18969/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.345; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"4",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard1",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"active",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18965_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18965/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.383; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.384; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.383; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.384; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.383; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.383; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.422; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.429; org.apache.solr.handler.admin.CoreAdminHandler; Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:47.429; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:18969_jj&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:48.424; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.426; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.431; org.apache.solr.cloud.RecoveryStrategy; Attempting to PeerSync from http://127.0.0.1:18962/jj/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.431; org.apache.solr.update.PeerSync; PeerSync: core=collection1 url=http://127.0.0.1:18969/jj START replicas=[http://127.0.0.1:18962/jj/collection1/] nUpdates=100
[junit4:junit4]   1> WARN  - 2013-04-15 03:45:49.431; org.apache.solr.update.PeerSync; no frame of reference to tell of we've missed updates
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.432; org.apache.solr.cloud.RecoveryStrategy; PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.432; org.apache.solr.cloud.RecoveryStrategy; Starting Replication Recovery. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.432; org.apache.solr.cloud.RecoveryStrategy; Begin buffering updates. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.433; org.apache.solr.update.UpdateLog; Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.433; org.apache.solr.cloud.RecoveryStrategy; Attempting to replicate from http://127.0.0.1:18962/jj/collection1/. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.433; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.438; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.444; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.494; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4259d58d lockFactory=org.apache.lucene.store.NativeFSLockFactory@538e56b7),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.495; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.496; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4259d58d lockFactory=org.apache.lucene.store.NativeFSLockFactory@538e56b7),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4259d58d lockFactory=org.apache.lucene.store.NativeFSLockFactory@538e56b7),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.496; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.497; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@254cd7b2 realtime
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.497; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.498; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 54
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.498; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for  10000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.499; org.apache.solr.handler.SnapPuller;  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.500; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for  10000
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.500; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.501; org.apache.solr.handler.SnapPuller; Master's generation: 2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.501; org.apache.solr.handler.SnapPuller; Slave's generation: 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.501; org.apache.solr.handler.SnapPuller; Starting replication process
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.503; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.503; org.apache.solr.handler.SnapPuller; Number of files in latest index in master: 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.504; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4/index.20130415104549504
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.505; org.apache.solr.handler.SnapPuller; Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@220ce59a lockFactory=org.apache.lucene.store.NativeFSLockFactory@202a029) fullCopy=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.507; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.508; org.apache.solr.handler.SnapPuller; Total time taken for download : 0 secs
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.508; org.apache.solr.update.DefaultSolrCoreState; Creating new IndexWriter...
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.508; org.apache.solr.update.DefaultSolrCoreState; Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.558; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16e3da96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1086cdc2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16e3da96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1086cdc2),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.558; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.559; org.apache.solr.update.DefaultSolrCoreState; New IndexWriter is ready to be used.
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.559; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@3ab241de main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.560; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@3ab241de main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.560; org.apache.solr.core.CachingDirectoryFactory; looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4/index.20130415104549504 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4/index.20130415104549504;done=true>>]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.560; org.apache.solr.core.CachingDirectoryFactory; Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4/index.20130415104549504
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.560; org.apache.solr.core.CachingDirectoryFactory; Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076/jetty4/index.20130415104549504
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.561; org.apache.solr.cloud.RecoveryStrategy; No replay needed. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.561; org.apache.solr.cloud.RecoveryStrategy; Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.561; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.561; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:49.562; org.apache.solr.cloud.RecoveryStrategy; Finished recovery process. core=collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:50.418; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:50.419; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4]   1>   "operation":"state",
[junit4:junit4]   1>   "core_node_name":"5",
[junit4:junit4]   1>   "numShards":"2",
[junit4:junit4]   1>   "shard":"shard2",
[junit4:junit4]   1>   "roles":null,
[junit4:junit4]   1>   "state":"active",
[junit4:junit4]   1>   "shard_state":"active",
[junit4:junit4]   1>   "core":"collection1",
[junit4:junit4]   1>   "collection":"collection1",
[junit4:junit4]   1>   "shard_range":null,
[junit4:junit4]   1>   "node_name":"127.0.0.1:18969_jj",
[junit4:junit4]   1>   "base_url":"http://127.0.0.1:18969/jj"}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:50.469; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:50.480; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:50.480; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:50.480; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:50.480; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:50.480; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:50.480; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.471; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.472; org.apache.solr.cloud.AbstractDistribZkTestBase; Recoveries finished - collection: collection1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.481; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.504; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@51cc7fbc lockFactory=org.apache.lucene.store.NativeFSLockFactory@495d56fd),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.505; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.506; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@51cc7fbc lockFactory=org.apache.lucene.store.NativeFSLockFactory@495d56fd),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@51cc7fbc lockFactory=org.apache.lucene.store.NativeFSLockFactory@495d56fd),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.506; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.507; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@2c89edf9 main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.507; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.508; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@2c89edf9 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.509; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 28
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.519; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.520; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4259d58d lockFactory=org.apache.lucene.store.NativeFSLockFactory@538e56b7),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4259d58d lockFactory=org.apache.lucene.store.NativeFSLockFactory@538e56b7),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.521; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.521; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@2266495a main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.522; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.522; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@2266495a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.523; org.apache.solr.update.SolrCmdDistributor; Distrib commit to:[StdNode: http://127.0.0.1:18959/jj/collection1/, StdNode: http://127.0.0.1:18965/jj/collection1/, StdNode: http://127.0.0.1:18969/jj/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.525; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.526; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2506007a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d980059),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2506007a lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d980059),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.527; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.527; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@491bcad7 main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.527; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.528; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@491bcad7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.528; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.529; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.530; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6ba65089 lockFactory=org.apache.lucene.store.NativeFSLockFactory@415c85ad),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6ba65089 lockFactory=org.apache.lucene.store.NativeFSLockFactory@415c85ad),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.531; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.531; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@3242214f main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.531; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.532; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@3242214f main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.532; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.532; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.533; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16e3da96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1086cdc2),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   1> 	commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16e3da96 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1086cdc2),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.533; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.534; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@64ff2225 main
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.534; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.535; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@64ff2225 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.535; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.536; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.537; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper... 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.539; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.541; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.542; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:51.544; org.apache.solr.core.SolrCore; [collection1] webapp=/jj path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.553; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1432352250708099072)} 0 6
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.569; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={update.distrib=FROMLEADER&_version_=-1432352250716487680&update.from=http://127.0.0.1:18959/jj/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1432352250716487680)} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.594; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={update.distrib=FROMLEADER&_version_=-1432352250725924864&update.from=http://127.0.0.1:18962/jj/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1432352250725924864)} 0 24
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.594; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1432352250725924864)} 0 30
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.595; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1432352250716487680)} 0 40
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.599; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[0 (1432352250761576448)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.605; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1432352250764722176)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.605; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[0 (1432352250764722176)]} 0 4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.608; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[1 (1432352250772062208)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.613; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1432352250775207936)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.614; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[1 (1432352250775207936)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.616; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[2 (1432352250780450816)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.623; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1432352250783596544)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.623; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[2 (1432352250783596544)]} 0 4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.626; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[3 (1432352250790936576)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.633; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1432352250796179456)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.634; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1432352250796179456)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.634; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[3]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.637; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[4 (1432352250802470912)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.645; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1432352250807713792)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.646; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1432352250807713792)]} 0 4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.646; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[4]} 0 6
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.649; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[5 (1432352250815053824)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.655; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1432352250818199552)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.655; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[5 (1432352250818199552)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.658; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[6 (1432352250824491008)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.663; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1432352250827636736)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.664; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[6 (1432352250827636736)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.666; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[7 (1432352250832879616)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.671; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1432352250836025344)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.672; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[7 (1432352250836025344)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.674; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[8 (1432352250841268224)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.679; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1432352250844413952)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.681; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[8 (1432352250844413952)]} 0 4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.683; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[9 (1432352250850705408)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.688; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1432352250853851136)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.689; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[9 (1432352250853851136)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.691; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[10 (1432352250859094016)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.698; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1432352250864336896)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.699; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1432352250864336896)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.699; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[10]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.702; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[11 (1432352250870628352)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.707; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1432352250873774080)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.708; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[11 (1432352250873774080)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.710; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[12 (1432352250879016960)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.715; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1432352250882162688)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.716; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[12 (1432352250882162688)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.718; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[13 (1432352250887405568)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.724; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1432352250890551296)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.725; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[13 (1432352250890551296)]} 0 4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.727; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[14 (1432352250896842752)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.734; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1432352250902085632)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.735; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1432352250902085632)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.736; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[14]} 0 6
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.738; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[15 (1432352250908377088)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.745; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1432352250913619968)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.746; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1432352250913619968)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.746; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[15]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.749; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[16 (1432352250919911424)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.755; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1432352250923057152)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.756; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[16 (1432352250923057152)]} 0 4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.758; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[17 (1432352250929348608)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.765; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1432352250933542912)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.766; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1432352250933542912)]} 0 4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.767; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[17]} 0 7
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.770; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[18 (1432352250941931520)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.775; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1432352250945077248)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.776; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[18 (1432352250945077248)]} 0 2
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.778; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[19 (1432352250950320128)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.782; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1432352250952417280)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.783; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[19 (1432352250952417280)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.786; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[20 (1432352250957660160)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.793; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1432352250963951616)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.794; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[20 (1432352250963951616)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.794; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[20]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.796; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[21 (1432352250969194496)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.803; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1432352250973388800)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.804; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1432352250973388800)]} 0 4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.804; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[21]} 0 6
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.806; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[22 (1432352250979680256)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.812; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1432352250983874560)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.813; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[22 (1432352250983874560)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.813; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[22]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.815; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[23 (1432352250989117440)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.822; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1432352250994360320)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.823; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[23 (1432352250994360320)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.823; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[23]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.826; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[24 (1432352251000651776)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.833; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1432352251005894656)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.834; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[24 (1432352251005894656)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.834; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[24]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.836; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[25 (1432352251011137536)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.842; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1432352251015331840)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.843; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[25 (1432352251015331840)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.843; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[25]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.846; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[26 (1432352251021623296)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.850; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1432352251023720448)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.851; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[26 (1432352251023720448)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.853; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[27 (1432352251028963328)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.860; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1432352251034206208)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.861; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[27 (1432352251034206208)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.861; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[27]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.864; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[28 (1432352251039449088)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.871; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1432352251045740544)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.872; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1432352251045740544)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.872; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[28]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.875; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[29 (1432352251052032000)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.881; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1432352251056226304)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.882; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[29 (1432352251056226304)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.882; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[29]} 0 5
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.885; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[30 (1432352251062517760)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.890; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1432352251064614912)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.891; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[30 (1432352251064614912)]} 0 4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.893; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[31 (1432352251070906368)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.901; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18962/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1432352251076149248)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.902; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1432352251076149248)]} 0 4
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.902; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[31]} 0 6
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.906; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[32 (1432352251083489280)]} 0 1
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.910; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={distrib.from=http://127.0.0.1:18959/jj/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1432352251086635008)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.911; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[32 (1432352251086635008)]} 0 3
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.913; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/jj path=/update params={wt=javabin&version=2} {add=[33 (1432352251091877888)]} 0 0
[junit4:junit4]   1> INFO  - 2013-04-15 03:45:53.918; org.apache.solr.update.processor.Lo

[...truncated too long message...]

rg.apache.solr.SolrTestCaseJ4; ###Ending testDistribSearch
[junit4:junit4]   1> INFO  - 2013-04-15 03:49:09.501; org.apache.solr.cloud.ZkTestServer; connecting to 127.0.0.1:18952 18952
[junit4:junit4]   1> INFO  - 2013-04-15 03:49:09.777; org.apache.solr.cloud.ChaosMonkey; monkey: stop shard! 18955
[junit4:junit4]   1> INFO  - 2013-04-15 03:49:09.784; org.apache.solr.SolrTestCaseJ4; ###Ending testDistribSearch
[junit4:junit4]   1> INFO  - 2013-04-15 03:49:09.785; org.apache.solr.cloud.ZkTestServer; connecting to 127.0.0.1:18952 18952
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyShardSplitTest-1365997526076
[junit4:junit4]   2> check const of shard1
[junit4:junit4]   2> client0
[junit4:junit4]   2> PROPS:2:{"shard":"shard1","state":"down","core":"collection1","collection":"collection1","node_name":"127.0.0.1:18959_jj","base_url":"http://127.0.0.1:18959/jj"}
[junit4:junit4]   2> error contacting client: IOException occured when talking to server at: http://127.0.0.1:18959/jj/collection1
[junit4:junit4]   2> 
[junit4:junit4]   2> client1
[junit4:junit4]   2> PROPS:4:{"shard":"shard1","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:18965_jj","base_url":"http://127.0.0.1:18965/jj"}
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:56
[junit4:junit4]   2> 
[junit4:junit4]   2> check const of shard2
[junit4:junit4]   2> client0
[junit4:junit4]   2> PROPS:3:{"shard":"shard2","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:18962_jj","base_url":"http://127.0.0.1:18962/jj","leader":"true"}
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:44
[junit4:junit4]   2> 
[junit4:junit4]   2> client1
[junit4:junit4]   2> PROPS:5:{"shard":"shard2","state":"active","core":"collection1","collection":"collection1","node_name":"127.0.0.1:18969_jj","base_url":"http://127.0.0.1:18969/jj"}
[junit4:junit4]   2>  live:true
[junit4:junit4]   2>  num:44
[junit4:junit4]   2> 
[junit4:junit4]   2> !!!! WARNING: best effort to remove /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365997526076 FAILED !!!!!
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeyShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=32D28CFFA5436A68 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=zh_SG -Dtests.timezone=Asia/Novokuznetsk -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE  224s J1 | ChaosMonkeyShardSplitTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: Wrong doc count on shard1_1 expected:<322> but was:<323>
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([32D28CFFA5436A68:B33402E7D21C0A54]:0)
[junit4:junit4]    > 	at org.apache.solr.cloud.ShardSplitTest.checkDocCountsAndShardStates(ShardSplitTest.java:155)
[junit4:junit4]    > 	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:154)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]   1> INFO  - 2013-04-15 03:49:09.807; org.apache.solr.SolrTestCaseJ4; ###deleteCore
[junit4:junit4]   2> 223745 T3970 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {timestamp=Pulsing41(freqCutoff=16 minBlockSize=35 maxBlockSize=157), range_facet_l=Pulsing41(freqCutoff=16 minBlockSize=35 maxBlockSize=157), id=PostingsFormat(name=NestedPulsing), range_facet_sl=PostingsFormat(name=Memory doPackFST= false), range_facet_si=PostingsFormat(name=MockSep), _version_=Pulsing41(freqCutoff=16 minBlockSize=35 maxBlockSize=157), multiDefault=PostingsFormat(name=MockSep), rnd_b=PostingsFormat(name=NestedPulsing), intDefault=Pulsing41(freqCutoff=16 minBlockSize=35 maxBlockSize=157)}, docValues:{}, sim=DefaultSimilarity, locale=zh_SG, timezone=Asia/Novokuznetsk
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Sun Microsystems Inc. 1.6.0_32 (64-bit)/cpus=16,threads=2,free=261827640,total=468320256
[junit4:junit4]   2> NOTE: All tests run in this JVM: [NumericFieldsTest, TimeZoneUtilsTest, SolrCoreCheckLockOnStartupTest, OpenExchangeRatesOrgProviderTest, FullSolrCloudDistribCmdsTest, TestCollationField, ZkSolrClientTest, TestDefaultSimilarityFactory, UniqFieldsUpdateProcessorFactoryTest, TestArbitraryIndexDir, NotRequiredUniqueKeyTest, TestRTGBase, SpellCheckCollatorTest, TestCodecSupport, TestJmxMonitoredMap, ConvertedLegacyTest, TestComponentsName, PreAnalyzedUpdateProcessorTest, TestMergePolicyConfig, JSONWriterTest, TestFastOutputStream, BadIndexSchemaTest, TestDFRSimilarityFactory, TestSolr4Spatial, TestReload, TestWordDelimiterFilterFactory, TestDistributedSearch, TestCoreDiscovery, EchoParamsTest, TestUniqueKeyFieldResource, NoCacheHeaderTest, StatsComponentTest, TestIndexSearcher, TestRandomFaceting, SignatureUpdateProcessorFactoryTest, SolrRequestParserTest, TestRangeQuery, TestCopyFieldCollectionResource, TermVectorComponentDistributedTest, BasicZkTest, SortByFunctionTest, TestSolrQueryParserResource, IndexReaderFactoryTest, SuggesterWFSTTest, IndexSchemaRuntimeFieldTest, DocValuesMultiTest, BasicDistributedZk2Test, TestAtomicUpdateErrorCases, TestOmitPositions, TestFoldingMultitermQuery, DocumentAnalysisRequestHandlerTest, CurrencyFieldOpenExchangeTest, TestShardHandlerFactory, TestCSVLoader, StandardRequestHandlerTest, SpellCheckComponentTest, TestZkChroot, SpellPossibilityIteratorTest, DOMUtilTest, TestFieldTypeResource, PathHierarchyTokenizerFactoryTest, TestSolrJ, TestAnalyzedSuggestions, TestIndexingPerformance, OpenCloseCoreStressTest, HighlighterConfigTest, TestRecovery, CollectionsAPIDistributedZkTest, TestMultiCoreConfBootstrap, RequestHandlersTest, SimplePostToolTest, ChaosMonkeySafeLeaderTest, SyncSliceTest, ShardRoutingCustomTest, TestDistributedGrouping, TestStressReorder, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, TestCoreContainer, SimpleFacetsTest, SolrCoreTest, TestGroupingSearch, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, OverseerCollectionProcessorTest, HighlighterTest, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, TestExtendedDismaxParser, SuggesterFSTTest, DocValuesTest, SuggesterTSTTest, SuggesterTest, TestStressLucene, TestTrie, SchemaVersionSpecificBehaviorTest, TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, TestRemoteStreaming, DistanceFunctionTest, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, TestQueryTypes, PrimitiveFieldTypeTest, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, MoreLikeThisHandlerTest, RequiredFieldsTest, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, SolrPluginUtilsTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, JsonLoaderTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, CSVRequestHandlerTest, SearchHandlerTest, CoreContainerCoreInitFailuresTest, TestBinaryResponseWriter, TestPropInject, TestQuerySenderListener, UpdateParamsTest, TestSolrIndexConfig, TestQuerySenderNoQuery, CopyFieldTest, SolrIndexConfigTest, BadComponentTest, TestStressRecovery, TestSolrDeletionPolicy2, MultiTermTest, SampleTest, TestBinaryField, TestSearchPerf, TestElisionMultitermQuery, TestConfig, OutputWriterTest, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, DirectSolrConnectionTest, TestLuceneMatchVersion, TestSweetSpotSimilarityFactory, TestNumberUtils, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TestBM25SimilarityFactory, ResourceLoaderTest, ScriptEngineTest, URLClassifyProcessorTest, TestFastLRUCache, DateMathParserTest, TestSolrXMLSerializer, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, ClusterStateTest, TestLRUCache, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SystemInfoHandlerTest, FileUtilsTest, CircularListTest, SolrTestCaseJ4Test, AliasIntegrationTest, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J1 in 224.03s, 1 test, 1 failure <<< FAILURES!

[...truncated 51 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/solr/common-build.xml:438: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/common-build.xml:1234: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java6/lucene/common-build.xml:878: There were test failures: 290 suites, 1204 tests, 1 failure, 27 ignored (8 assumptions)

Total time: 57 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message