lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1328 - Still Failing
Date Thu, 13 Jun 2013 17:01:03 GMT
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1328/

1 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:41428/y_q returned non ok status:500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:41428/y_q returned non ok status:500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([94C1FB14CE220E9A:1527750CB97D6EA6]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9387 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4]   2> 539211 T860 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /y_q/
[junit4:junit4]   2> 539218 T860 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeyShardSplitTest-1371141565911
[junit4:junit4]   2> 539220 T860 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 539220 T861 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 539321 T860 oasc.ZkTestServer.run start zk server on port:55935
[junit4:junit4]   2> 539322 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 539326 T867 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1426241c name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 539327 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 539327 T860 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 539341 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 539342 T869 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@605d5c9c name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 539343 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 539343 T860 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 539352 T860 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 539355 T860 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 539358 T860 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 539367 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 539368 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 539380 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 539381 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 539485 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 539486 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 539495 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 539496 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 539499 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 539499 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 539503 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 539503 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 539507 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 539507 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 539511 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 539511 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 539514 T860 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 539515 T860 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 539771 T860 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 539775 T860 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:27612
[junit4:junit4]   2> 539776 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 539777 T860 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 539777 T860 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211
[junit4:junit4]   2> 539792 T860 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/solr.xml
[junit4:junit4]   2> 539793 T860 oasc.CoreContainer.<init> New CoreContainer 1156464092
[junit4:junit4]   2> 539794 T860 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/'
[junit4:junit4]   2> 539794 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/'
[junit4:junit4]   2> 539924 T860 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 539925 T860 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 539926 T860 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 539927 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 539927 T860 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 539928 T860 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 539928 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 539929 T860 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 539929 T860 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 539930 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 539949 T860 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 539950 T860 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55935/solr
[junit4:junit4]   2> 539951 T860 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 539952 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 539955 T880 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5005e58 name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 539956 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 539959 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 539972 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 539975 T882 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43256374 name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 539975 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 539977 T860 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 539987 T860 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 539990 T860 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 539997 T860 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:27612_y_q
[junit4:junit4]   2> 539999 T860 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:27612_y_q
[junit4:junit4]   2> 540002 T860 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 540018 T860 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 540026 T860 oasc.Overseer.start Overseer (id=89859133664067587-127.0.0.1:27612_y_q-n_0000000000) starting
[junit4:junit4]   2> 540036 T860 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 540046 T884 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 540047 T860 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 540049 T860 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 540057 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 540059 T883 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 540064 T885 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 540064 T885 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 541564 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 541565 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:27612_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:27612/y_q"}
[junit4:junit4]   2> 541565 T883 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 541565 T883 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 541611 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 542068 T885 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/collection1
[junit4:junit4]   2> 542068 T885 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 542069 T885 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 542069 T885 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 542071 T885 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/collection1/'
[junit4:junit4]   2> 542072 T885 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/collection1/lib/README' to classloader
[junit4:junit4]   2> 542073 T885 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 542127 T885 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 542189 T885 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 542291 T885 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 542297 T885 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 542871 T885 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 542875 T885 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 542878 T885 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 542883 T885 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 542913 T885 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 542914 T885 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1371141566211/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data/
[junit4:junit4]   2> 542914 T885 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@52f34d18
[junit4:junit4]   2> 542915 T885 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 542915 T885 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data
[junit4:junit4]   2> 542916 T885 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data/index/
[junit4:junit4]   2> 542916 T885 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 542917 T885 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data/index
[junit4:junit4]   2> 542922 T885 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5cd85e80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@668d9c17),segFN=segments_1,generation=1}
[junit4:junit4]   2> 542922 T885 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 542925 T885 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 542926 T885 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 542927 T885 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 542927 T885 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 542928 T885 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 542928 T885 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 542929 T885 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 542930 T885 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 542930 T885 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 542946 T885 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 542954 T885 oass.SolrIndexSearcher.<init> Opening Searcher@1b263d21 main
[junit4:junit4]   2> 542955 T885 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/control/data/tlog
[junit4:junit4]   2> 542956 T885 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 542956 T885 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 542960 T886 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b263d21 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 542963 T885 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 542963 T885 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:27612/y_q collection:control_collection shard:shard1
[junit4:junit4]   2> 542964 T885 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 542981 T885 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 542991 T885 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 542991 T885 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 542992 T885 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:27612/y_q/collection1/
[junit4:junit4]   2> 542992 T885 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 542992 T885 oasc.SyncStrategy.syncToMe http://127.0.0.1:27612/y_q/collection1/ has no replicas
[junit4:junit4]   2> 542993 T885 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:27612/y_q/collection1/
[junit4:junit4]   2> 542993 T885 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 543115 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 543155 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 543212 T885 oasc.ZkController.register We are http://127.0.0.1:27612/y_q/collection1/ and leader is http://127.0.0.1:27612/y_q/collection1/
[junit4:junit4]   2> 543212 T885 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:27612/y_q
[junit4:junit4]   2> 543212 T885 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 543213 T885 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 543213 T885 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 543215 T885 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 543216 T860 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 543217 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 543218 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 543236 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 543238 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 543240 T889 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ec22862 name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 543241 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 543243 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 543245 T860 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 543507 T860 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 543511 T860 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41428
[junit4:junit4]   2> 543512 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 543513 T860 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 543513 T860 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938
[junit4:junit4]   2> 543514 T860 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/solr.xml
[junit4:junit4]   2> 543514 T860 oasc.CoreContainer.<init> New CoreContainer 1618698809
[junit4:junit4]   2> 543515 T860 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/'
[junit4:junit4]   2> 543516 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/'
[junit4:junit4]   2> 543645 T860 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 543646 T860 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 543646 T860 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 543647 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 543648 T860 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 543648 T860 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 543649 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 543649 T860 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 543650 T860 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 543651 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 543669 T860 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 543670 T860 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55935/solr
[junit4:junit4]   2> 543670 T860 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 543672 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 543675 T900 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6edb135a name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 543676 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 543691 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 543704 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 543706 T902 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72f46074 name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 543707 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 543716 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 544666 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 544667 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"1",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:27612_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:27612/y_q"}
[junit4:junit4]   2> 544682 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 544682 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 544682 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 544720 T860 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41428_y_q
[junit4:junit4]   2> 544722 T860 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41428_y_q
[junit4:junit4]   2> 544724 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 544724 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 544724 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 544726 T882 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 544733 T902 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 544733 T889 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 544738 T903 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 544738 T903 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 546187 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 546188 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:41428_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:41428/y_q"}
[junit4:junit4]   2> 546189 T883 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 546189 T883 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 546209 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 546209 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 546209 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 546742 T903 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/collection1
[junit4:junit4]   2> 546742 T903 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 546743 T903 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 546743 T903 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 546745 T903 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/collection1/'
[junit4:junit4]   2> 546747 T903 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 546747 T903 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/collection1/lib/README' to classloader
[junit4:junit4]   2> 546802 T903 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 546863 T903 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 546965 T903 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 546971 T903 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 547546 T903 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 547551 T903 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 547553 T903 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 547559 T903 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 547589 T903 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 547589 T903 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1371141569938/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1/
[junit4:junit4]   2> 547590 T903 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@52f34d18
[junit4:junit4]   2> 547591 T903 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 547591 T903 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1
[junit4:junit4]   2> 547592 T903 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1/index/
[junit4:junit4]   2> 547592 T903 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 547593 T903 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1/index
[junit4:junit4]   2> 547597 T903 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_1,generation=1}
[junit4:junit4]   2> 547597 T903 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 547601 T903 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 547601 T903 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 547602 T903 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 547603 T903 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 547604 T903 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 547604 T903 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 547605 T903 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 547605 T903 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 547606 T903 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 547622 T903 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 547629 T903 oass.SolrIndexSearcher.<init> Opening Searcher@7bd7d288 main
[junit4:junit4]   2> 547629 T903 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty1/tlog
[junit4:junit4]   2> 547630 T903 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 547631 T903 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 547635 T904 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7bd7d288 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 547638 T903 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 547638 T903 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41428/y_q collection:collection1 shard:shard1
[junit4:junit4]   2> 547639 T903 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 547662 T903 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 547673 T903 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 547673 T903 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 547674 T903 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41428/y_q/collection1/
[junit4:junit4]   2> 547674 T903 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 547674 T903 oasc.SyncStrategy.syncToMe http://127.0.0.1:41428/y_q/collection1/ has no replicas
[junit4:junit4]   2> 547675 T903 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41428/y_q/collection1/
[junit4:junit4]   2> 547675 T903 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 547713 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 547729 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 547729 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 547729 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 547732 T903 oasc.ZkController.register We are http://127.0.0.1:41428/y_q/collection1/ and leader is http://127.0.0.1:41428/y_q/collection1/
[junit4:junit4]   2> 547732 T903 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41428/y_q
[junit4:junit4]   2> 547732 T903 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 547733 T903 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 547733 T903 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 547734 T903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 547735 T860 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 547736 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 547737 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 548001 T860 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 548004 T860 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:14096
[junit4:junit4]   2> 548005 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 548005 T860 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 548006 T860 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446
[junit4:junit4]   2> 548006 T860 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/solr.xml
[junit4:junit4]   2> 548007 T860 oasc.CoreContainer.<init> New CoreContainer 1138398660
[junit4:junit4]   2> 548008 T860 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/'
[junit4:junit4]   2> 548008 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/'
[junit4:junit4]   2> 548128 T860 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 548129 T860 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 548130 T860 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 548130 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 548131 T860 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 548132 T860 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 548132 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 548133 T860 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 548133 T860 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 548134 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 548151 T860 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 548152 T860 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55935/solr
[junit4:junit4]   2> 548152 T860 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 548153 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 548157 T916 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@681892be name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 548157 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 548172 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 548186 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 548188 T918 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ab921e9 name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 548188 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 548194 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 549198 T860 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:14096_y_q
[junit4:junit4]   2> 549200 T860 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:14096_y_q
[junit4:junit4]   2> 549203 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 549203 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 549203 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 549204 T918 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 549214 T882 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 549214 T902 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 549214 T889 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 549219 T919 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 549220 T919 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 549234 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 549235 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:41428_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:41428/y_q"}
[junit4:junit4]   2> 549239 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:14096_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:14096/y_q"}
[junit4:junit4]   2> 549240 T883 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 549240 T883 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 549251 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 549251 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 549251 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 549251 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 550222 T919 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/collection1
[junit4:junit4]   2> 550222 T919 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 550223 T919 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 550224 T919 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 550225 T919 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/collection1/'
[junit4:junit4]   2> 550227 T919 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/collection1/lib/README' to classloader
[junit4:junit4]   2> 550228 T919 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 550282 T919 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 550343 T919 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 550445 T919 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 550451 T919 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 551029 T919 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 551034 T919 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 551036 T919 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 551041 T919 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 551071 T919 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 551072 T919 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1371141574446/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2/
[junit4:junit4]   2> 551072 T919 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@52f34d18
[junit4:junit4]   2> 551073 T919 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 551074 T919 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2
[junit4:junit4]   2> 551074 T919 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2/index/
[junit4:junit4]   2> 551075 T919 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 551075 T919 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2/index
[junit4:junit4]   2> 551079 T919 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_1,generation=1}
[junit4:junit4]   2> 551080 T919 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 551084 T919 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 551084 T919 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 551085 T919 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 551086 T919 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 551086 T919 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 551087 T919 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 551087 T919 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 551088 T919 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 551089 T919 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 551105 T919 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 551113 T919 oass.SolrIndexSearcher.<init> Opening Searcher@3b20df58 main
[junit4:junit4]   2> 551114 T919 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty2/tlog
[junit4:junit4]   2> 551115 T919 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 551115 T919 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 551121 T920 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b20df58 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 551123 T919 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 551124 T919 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:14096/y_q collection:collection1 shard:shard2
[junit4:junit4]   2> 551125 T919 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 551134 T919 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 551151 T919 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 551151 T919 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 551151 T919 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:14096/y_q/collection1/
[junit4:junit4]   2> 551152 T919 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 551152 T919 oasc.SyncStrategy.syncToMe http://127.0.0.1:14096/y_q/collection1/ has no replicas
[junit4:junit4]   2> 551153 T919 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:14096/y_q/collection1/
[junit4:junit4]   2> 551153 T919 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 552267 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 552292 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 552292 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 552292 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 552292 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 552331 T919 oasc.ZkController.register We are http://127.0.0.1:14096/y_q/collection1/ and leader is http://127.0.0.1:14096/y_q/collection1/
[junit4:junit4]   2> 552331 T919 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:14096/y_q
[junit4:junit4]   2> 552331 T919 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 552332 T919 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 552332 T919 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 552334 T919 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 552335 T860 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 552336 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 552337 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 552622 T860 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 552626 T860 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:62951
[junit4:junit4]   2> 552627 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 552628 T860 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 552628 T860 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047
[junit4:junit4]   2> 552629 T860 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/solr.xml
[junit4:junit4]   2> 552629 T860 oasc.CoreContainer.<init> New CoreContainer 173336716
[junit4:junit4]   2> 552630 T860 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/'
[junit4:junit4]   2> 552631 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/'
[junit4:junit4]   2> 552761 T860 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 552762 T860 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 552762 T860 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 552763 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 552764 T860 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 552764 T860 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 552765 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 552765 T860 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 552766 T860 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 552767 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 552785 T860 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 552786 T860 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55935/solr
[junit4:junit4]   2> 552787 T860 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 552788 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 552791 T932 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e8d50cd name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 552792 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 552808 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 552822 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 552825 T934 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34a846c9 name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 552825 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 552834 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 553805 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 553806 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"3",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:14096_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:14096/y_q"}
[junit4:junit4]   2> 553822 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 553822 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 553822 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 553822 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 553822 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 553838 T860 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62951_y_q
[junit4:junit4]   2> 553840 T860 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:62951_y_q
[junit4:junit4]   2> 553843 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 553843 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 553843 T918 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 553843 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 553843 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 553843 T934 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 553844 T882 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 553844 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 553850 T902 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 553851 T889 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 553856 T935 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 553856 T935 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 555327 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 555329 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4]   2> 555329 T883 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 555329 T883 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4]   2> 555345 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 555345 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 555345 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 555345 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 555345 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 555860 T935 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/collection1
[junit4:junit4]   2> 555860 T935 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 555861 T935 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 555862 T935 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 555863 T935 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/collection1/'
[junit4:junit4]   2> 555865 T935 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/collection1/lib/README' to classloader
[junit4:junit4]   2> 555866 T935 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 555920 T935 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 555982 T935 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 556084 T935 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 556090 T935 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> 556645 T935 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 556648 T935 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 556650 T935 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 556654 T935 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 556677 T935 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 556678 T935 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1371141579047/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/
[junit4:junit4]   2> 556678 T935 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@52f34d18
[junit4:junit4]   2> 556678 T935 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 556679 T935 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3
[junit4:junit4]   2> 556679 T935 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index/
[junit4:junit4]   2> 556679 T935 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 556680 T935 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index
[junit4:junit4]   2> 556683 T935 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cc1076 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45f28528),segFN=segments_1,generation=1}
[junit4:junit4]   2> 556683 T935 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 556686 T935 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 556686 T935 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 556687 T935 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 556687 T935 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 556688 T935 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 556688 T935 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 556688 T935 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 556689 T935 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 556689 T935 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 556700 T935 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 556705 T935 oass.SolrIndexSearcher.<init> Opening Searcher@6ba1d1c8 main
[junit4:junit4]   2> 556705 T935 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/tlog
[junit4:junit4]   2> 556706 T935 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 556706 T935 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 556710 T936 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6ba1d1c8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 556713 T935 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 556713 T935 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:62951/y_q collection:collection1 shard:shard1
[junit4:junit4]   2> 556716 T935 oasc.ZkController.register We are http://127.0.0.1:62951/y_q/collection1/ and leader is http://127.0.0.1:41428/y_q/collection1/
[junit4:junit4]   2> 556716 T935 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:62951/y_q
[junit4:junit4]   2> 556716 T935 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 556717 T935 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C157 name=collection1 org.apache.solr.core.SolrCore@3fe058d8 url=http://127.0.0.1:62951/y_q/collection1 node=127.0.0.1:62951_y_q C157_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:62951_y_q, base_url=http://127.0.0.1:62951/y_q}
[junit4:junit4]   2> 556717 T937 C157 P62951 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 556717 T935 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 556718 T937 C157 P62951 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 556718 T937 C157 P62951 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 556719 T937 C157 P62951 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 556719 T860 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 556720 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 556720 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 556721 T937 C157 P62951 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 556743 T892 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 556849 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 556850 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4]   2> 556868 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 556868 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 556868 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 556868 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 556868 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 557008 T860 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 557011 T860 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:32634
[junit4:junit4]   2> 557012 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 557012 T860 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 557013 T860 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434
[junit4:junit4]   2> 557014 T860 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/solr.xml
[junit4:junit4]   2> 557014 T860 oasc.CoreContainer.<init> New CoreContainer 1242169106
[junit4:junit4]   2> 557015 T860 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/'
[junit4:junit4]   2> 557015 T860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/'
[junit4:junit4]   2> 557145 T860 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 557145 T860 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 557146 T860 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 557147 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 557147 T860 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 557148 T860 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 557148 T860 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 557149 T860 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 557149 T860 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 557150 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 557169 T860 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 557169 T860 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55935/solr
[junit4:junit4]   2> 557170 T860 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 557171 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 557174 T949 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c443ec6 name:ZooKeeperConnection Watcher:127.0.0.1:55935 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 557175 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 557177 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 557186 T860 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 557188 T951 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19c79ded name:ZooKeeperConnection Watcher:127.0.0.1:55935/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 557189 T860 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 557201 T860 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 557745 T892 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4]   2> 557745 T892 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:62951_y_q&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002 
[junit4:junit4]   2> 558204 T860 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32634_y_q
[junit4:junit4]   2> 558210 T860 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32634_y_q
[junit4:junit4]   2> 558213 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 558214 T934 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 558213 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 558213 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 558214 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 558214 T918 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 558214 T951 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 558216 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 558215 T882 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 558227 T902 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 558227 T889 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 558232 T952 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 558232 T952 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 558372 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 558373 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32634_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32634/y_q"}
[junit4:junit4]   2> 558374 T883 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 558374 T883 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4]   2> 558388 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 558388 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 558388 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 558388 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 558388 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 558388 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 559235 T952 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/collection1
[junit4:junit4]   2> 559235 T952 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 559236 T952 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 559236 T952 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 559238 T952 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/collection1/'
[junit4:junit4]   2> 559239 T952 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/collection1/lib/README' to classloader
[junit4:junit4]   2> 559240 T952 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 559293 T952 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 559367 T952 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 559469 T952 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 559475 T952 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4]   2> ASYNC  NEW_CORE C158 name=collection1 org.apache.solr.core.SolrCore@3fe058d8 url=http://127.0.0.1:62951/y_q/collection1 node=127.0.0.1:62951_y_q C158_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:62951_y_q, base_url=http://127.0.0.1:62951/y_q}
[junit4:junit4]   2> 559747 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:41428/y_q/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 559747 T937 C158 P62951 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:62951/y_q START replicas=[http://127.0.0.1:41428/y_q/collection1/] nUpdates=100
[junit4:junit4]   2> 559748 T937 C158 P62951 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 559749 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 559749 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 559749 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 559750 T937 C158 P62951 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 559750 T937 C158 P62951 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:41428/y_q/collection1/. core=collection1
[junit4:junit4]   2> 559750 T937 C158 P62951 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C159 name=collection1 org.apache.solr.core.SolrCore@61c2d960 url=http://127.0.0.1:41428/y_q/collection1 node=127.0.0.1:41428_y_q C159_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:41428_y_q, base_url=http://127.0.0.1:41428/y_q, leader=true}
[junit4:junit4]   2> 559757 T893 C159 P41428 oasc.SolrCore.execute [collection1] webapp=/y_q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 559764 T894 C159 P41428 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 559767 T894 C159 P41428 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_1,generation=1}
[junit4:junit4]   2> 559767 T894 C159 P41428 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 559768 T894 C159 P41428 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_2,generation=2}
[junit4:junit4]   2> 559769 T894 C159 P41428 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 559770 T894 C159 P41428 oass.SolrIndexSearcher.<init> Opening Searcher@3d4ef04f realtime
[junit4:junit4]   2> 559770 T894 C159 P41428 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 559771 T894 C159 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4]   2> 559772 T937 C158 P62951 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 559772 T937 C158 P62951 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 559774 T895 C159 P41428 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 559775 T895 C159 P41428 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 559776 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 559776 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 559777 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 559779 T895 C159 P41428 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 559780 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 559781 T937 C158 P62951 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index.20130613123946473
[junit4:junit4]   2> 559782 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@11697660 lockFactory=org.apache.lucene.store.NativeFSLockFactory@49b5ec72) fullCopy=false
[junit4:junit4]   2> 559784 T895 C159 P41428 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 559785 T937 C158 P62951 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 559786 T937 C158 P62951 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 559787 T937 C158 P62951 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 559789 T937 C158 P62951 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cc1076 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45f28528),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cc1076 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45f28528),segFN=segments_2,generation=2}
[junit4:junit4]   2> 559790 T937 C158 P62951 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 559790 T937 C158 P62951 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 559791 T937 C158 P62951 oass.SolrIndexSearcher.<init> Opening Searcher@3e84a226 main
[junit4:junit4]   2> 559792 T936 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3e84a226 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 559793 T937 C158 P62951 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index.20130613123946473 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index.20130613123946473;done=true>>]
[junit4:junit4]   2> 559793 T937 C158 P62951 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index.20130613123946473
[junit4:junit4]   2> 559794 T937 C158 P62951 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty3/index.20130613123946473
[junit4:junit4]   2> 559794 T937 C158 P62951 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 559794 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 559794 T937 C158 P62951 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 559795 T937 C158 P62951 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 559797 T937 C158 P62951 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 559894 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 559895 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"4",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4]   2> 559908 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 559909 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 559908 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 559908 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 559908 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 559908 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 560036 T952 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 560040 T952 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 560042 T952 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 560048 T952 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 560075 T952 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 560075 T952 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1371141583434/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/
[junit4:junit4]   2> 560076 T952 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@52f34d18
[junit4:junit4]   2> 560076 T952 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 560077 T952 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4
[junit4:junit4]   2> 560078 T952 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index/
[junit4:junit4]   2> 560078 T952 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 560079 T952 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index
[junit4:junit4]   2> 560083 T952 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f054dc5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4eebd02e),segFN=segments_1,generation=1}
[junit4:junit4]   2> 560083 T952 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 560087 T952 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 560087 T952 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 560088 T952 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 560089 T952 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 560090 T952 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 560090 T952 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 560091 T952 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 560092 T952 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 560092 T952 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 560109 T952 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 560116 T952 oass.SolrIndexSearcher.<init> Opening Searcher@b68096e main
[junit4:junit4]   2> 560117 T952 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/tlog
[junit4:junit4]   2> 560118 T952 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 560119 T952 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 560124 T955 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b68096e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 560126 T952 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 560127 T952 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:32634/y_q collection:collection1 shard:shard2
[junit4:junit4]   2> 560130 T952 oasc.ZkController.register We are http://127.0.0.1:32634/y_q/collection1/ and leader is http://127.0.0.1:14096/y_q/collection1/
[junit4:junit4]   2> 560130 T952 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32634/y_q
[junit4:junit4]   2> 560131 T952 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 560131 T952 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C160 name=collection1 org.apache.solr.core.SolrCore@5b967893 url=http://127.0.0.1:32634/y_q/collection1 node=127.0.0.1:32634_y_q C160_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:32634_y_q, base_url=http://127.0.0.1:32634/y_q}
[junit4:junit4]   2> 560132 T956 C160 P32634 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 560132 T952 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 560132 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 560133 T956 C160 P32634 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 560133 T956 C160 P32634 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 560133 T860 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 560134 T860 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 560135 T860 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 560135 T956 C160 P32634 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 560160 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 560161 T908 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 560161 T860 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 560162 T860 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 560163 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 561165 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 561413 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 561414 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32634_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32634/y_q"}
[junit4:junit4]   2> 561436 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 561436 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 561436 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 561436 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 561436 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 561436 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 562163 T908 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 562163 T908 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:32634_y_q&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002 
[junit4:junit4]   2> 562168 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 563170 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C160_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:32634_y_q, base_url=http://127.0.0.1:32634/y_q}
[junit4:junit4]   2> 564165 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:14096/y_q/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 564165 T956 C160 P32634 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32634/y_q START replicas=[http://127.0.0.1:14096/y_q/collection1/] nUpdates=100
[junit4:junit4]   2> 564166 T956 C160 P32634 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 564166 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 564166 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 564166 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 564167 T956 C160 P32634 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 564168 T956 C160 P32634 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:14096/y_q/collection1/. core=collection1
[junit4:junit4]   2> 564168 T956 C160 P32634 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 564172 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C161 name=collection1 org.apache.solr.core.SolrCore@56edda62 url=http://127.0.0.1:14096/y_q/collection1 node=127.0.0.1:14096_y_q C161_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:14096_y_q, base_url=http://127.0.0.1:14096/y_q, leader=true}
[junit4:junit4]   2> 564175 T909 C161 P14096 oasc.SolrCore.execute [collection1] webapp=/y_q path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 564181 T910 C161 P14096 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 564184 T910 C161 P14096 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_1,generation=1}
[junit4:junit4]   2> 564185 T910 C161 P14096 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 564186 T910 C161 P14096 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_2,generation=2}
[junit4:junit4]   2> 564186 T910 C161 P14096 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 564187 T910 C161 P14096 oass.SolrIndexSearcher.<init> Opening Searcher@3eecc8ce realtime
[junit4:junit4]   2> 564187 T910 C161 P14096 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 564188 T910 C161 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4]   2> 564189 T956 C160 P32634 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 564189 T956 C160 P32634 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 564191 T911 C161 P14096 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 564191 T911 C161 P14096 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 564192 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 564192 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 564193 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 564195 T911 C161 P14096 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0 
[junit4:junit4]   2> 564196 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 564197 T956 C160 P32634 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index.20130613123950889
[junit4:junit4]   2> 564198 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@322786b2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@41246224) fullCopy=false
[junit4:junit4]   2> 564201 T911 C161 P14096 oasc.SolrCore.execute [collection1] webapp=/y_q path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 564203 T956 C160 P32634 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 564204 T956 C160 P32634 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 564204 T956 C160 P32634 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 564206 T956 C160 P32634 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f054dc5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4eebd02e),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f054dc5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4eebd02e),segFN=segments_2,generation=2}
[junit4:junit4]   2> 564207 T956 C160 P32634 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 564207 T956 C160 P32634 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 564208 T956 C160 P32634 oass.SolrIndexSearcher.<init> Opening Searcher@1b83f973 main
[junit4:junit4]   2> 564209 T955 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b83f973 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 564210 T956 C160 P32634 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index.20130613123950889 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index.20130613123950889;done=true>>]
[junit4:junit4]   2> 564210 T956 C160 P32634 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index.20130613123950889
[junit4:junit4]   2> 564210 T956 C160 P32634 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index.20130613123950889
[junit4:junit4]   2> 564211 T956 C160 P32634 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 564211 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 564211 T956 C160 P32634 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 564212 T956 C160 P32634 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 564213 T956 C160 P32634 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 564442 T883 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 564443 T883 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"5",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32634_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32634/y_q"}
[junit4:junit4]   2> 564462 T882 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 564462 T918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 564462 T902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 564462 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 564462 T889 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 564462 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 565174 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 565175 T860 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C162 name=collection1 org.apache.solr.core.SolrCore@7a189a1 url=http://127.0.0.1:27612/y_q/collection1 node=127.0.0.1:27612_y_q C162_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:27612_y_q, base_url=http://127.0.0.1:27612/y_q, leader=true}
[junit4:junit4]   2> 565184 T872 C162 P27612 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 565188 T872 C162 P27612 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5cd85e80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@668d9c17),segFN=segments_1,generation=1}
[junit4:junit4]   2> 565189 T872 C162 P27612 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 565190 T872 C162 P27612 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5cd85e80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@668d9c17),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5cd85e80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@668d9c17),segFN=segments_2,generation=2}
[junit4:junit4]   2> 565190 T872 C162 P27612 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 565191 T872 C162 P27612 oass.SolrIndexSearcher.<init> Opening Searcher@c73606 main
[junit4:junit4]   2> 565192 T872 C162 P27612 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 565193 T886 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c73606 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 565193 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4]   2> ASYNC  NEW_CORE C163 name=collection1 org.apache.solr.core.SolrCore@56edda62 url=http://127.0.0.1:14096/y_q/collection1 node=127.0.0.1:14096_y_q C163_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:14096_y_q, base_url=http://127.0.0.1:14096/y_q, leader=true}
[junit4:junit4]   2> 565206 T912 C163 P14096 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 565207 T912 C163 P14096 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d597a48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ea6404c),segFN=segments_3,generation=3}
[junit4:junit4]   2> 565208 T912 C163 P14096 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 565209 T912 C163 P14096 oass.SolrIndexSearcher.<init> Opening Searcher@13479058 main
[junit4:junit4]   2> 565209 T912 C163 P14096 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 565210 T920 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13479058 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 565211 T912 C163 P14096 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:41428/y_q/collection1/, StdNode: http://127.0.0.1:62951/y_q/collection1/, StdNode: http://127.0.0.1:32634/y_q/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> ASYNC  NEW_CORE C164 name=collection1 org.apache.solr.core.SolrCore@61c2d960 url=http://127.0.0.1:41428/y_q/collection1 node=127.0.0.1:41428_y_q C164_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:41428_y_q, base_url=http://127.0.0.1:41428/y_q, leader=true}
[junit4:junit4]   2> 565214 T896 C164 P41428 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 565217 T896 C164 P41428 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@147409e6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fe14ce7),segFN=segments_3,generation=3}
[junit4:junit4]   2> 565217 T896 C164 P41428 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 565218 T896 C164 P41428 oass.SolrIndexSearcher.<init> Opening Searcher@60c3fd55 main
[junit4:junit4]   2> 565218 T896 C164 P41428 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 565219 T904 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60c3fd55 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 565220 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> ASYNC  NEW_CORE C165 name=collection1 org.apache.solr.core.SolrCore@5b967893 url=http://127.0.0.1:32634/y_q/collection1 node=127.0.0.1:32634_y_q C165_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:32634_y_q, base_url=http://127.0.0.1:32634/y_q}
[junit4:junit4]   2> 565223 T941 C165 P32634 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> ASYNC  NEW_CORE C166 name=collection1 org.apache.solr.core.SolrCore@3fe058d8 url=http://127.0.0.1:62951/y_q/collection1 node=127.0.0.1:62951_y_q C166_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:62951_y_q, base_url=http://127.0.0.1:62951/y_q}
[junit4:junit4]   2> 565223 T924 C166 P62951 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 565224 T941 C165 P32634 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f054dc5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4eebd02e),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f054dc5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4eebd02e),segFN=segments_3,generation=3}
[junit4:junit4]   2> 565224 T924 C166 P62951 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cc1076 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45f28528),segFN=segments_2,generation=2}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@25cc1076 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45f28528),segFN=segments_3,generation=3}
[junit4:junit4]   2> 565225 T941 C165 P32634 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 565225 T924 C166 P62951 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4]   2> 565226 T941 C165 P32634 oass.SolrIndexSearcher.<init> Opening Searcher@16386e42 main
[junit4:junit4]   2> 565226 T924 C166 P62951 oass.SolrIndexSearcher.<init> Opening Searcher@4df1c78c main
[junit4:junit4]   2> 565227 T941 C165 P32634 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 565227 T924 C166 P62951 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 565228 T955 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16386e42 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 565228 T936 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4df1c78c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 565229 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 565229 T924 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4]   2> 565230 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4]   2> 565232 T860 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 565235 T897 C164 P41428 oasc.SolrCore.execute [collection1] webapp=/y_q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 565237 T925 C166 P62951 oasc.SolrCore.execute [collection1] webapp=/y_q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 565239 T913 C163 P14096 oasc.SolrCore.execute [collection1] webapp=/y_q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 565242 T942 C165 P32634 oasc.SolrCore.execute [collection1] webapp=/y_q path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 567247 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437746168005132288)} 0 2
[junit4:junit4]   2> 567257 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={update.distrib=FROMLEADER&_version_=-1437746168009326592&update.from=http://127.0.0.1:41428/y_q/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437746168009326592)} 0 3
[junit4:junit4]   2> 567261 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={update.distrib=FROMLEADER&_version_=-1437746168014569472&update.from=http://127.0.0.1:14096/y_q/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1437746168014569472)} 0 2
[junit4:junit4]   2> 567261 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1437746168014569472)} 0 7
[junit4:junit4]   2> 567262 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1437746168009326592)} 0 12
[junit4:junit4]   2> 567267 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[0 (1437746168025055232)]} 0 2
[junit4:junit4]   2> 567274 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1437746168030298112)]} 0 1
[junit4:junit4]   2> 567275 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[0 (1437746168030298112)]} 0 5
[junit4:junit4]   2> 567279 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[1 (1437746168038686720)]} 0 1
[junit4:junit4]   2> 567286 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1437746168042881024)]} 0 1
[junit4:junit4]   2> 567286 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[1 (1437746168042881024)]} 0 4
[junit4:junit4]   2> 567290 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[2 (1437746168050221056)]} 0 1
[junit4:junit4]   2> 567299 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1437746168056512512)]} 0 1
[junit4:junit4]   2> 567301 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1437746168056512512)]} 0 5
[junit4:junit4]   2> 567301 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[2]} 0 9
[junit4:junit4]   2> 567305 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[3 (1437746168065949696)]} 0 1
[junit4:junit4]   2> 567314 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1437746168072241152)]} 0 1
[junit4:junit4]   2> 567314 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1437746168072241152)]} 0 4
[junit4:junit4]   2> 567315 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[3]} 0 7
[junit4:junit4]   2> 567319 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[4 (1437746168080629760)]} 0 1
[junit4:junit4]   2> 567326 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1437746168084824064)]} 0 1
[junit4:junit4]   2> 567326 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[4 (1437746168084824064)]} 0 4
[junit4:junit4]   2> 567330 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[5 (1437746168092164096)]} 0 1
[junit4:junit4]   2> 567336 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1437746168095309824)]} 0 1
[junit4:junit4]   2> 567336 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[5 (1437746168095309824)]} 0 4
[junit4:junit4]   2> 567340 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[6 (1437746168102649856)]} 0 1
[junit4:junit4]   2> 567349 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1437746168108941312)]} 0 1
[junit4:junit4]   2> 567349 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1437746168108941312)]} 0 4
[junit4:junit4]   2> 567350 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[6]} 0 7
[junit4:junit4]   2> 567354 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[7 (1437746168117329920)]} 0 1
[junit4:junit4]   2> 567361 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1437746168121524224)]} 0 1
[junit4:junit4]   2> 567361 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[7 (1437746168121524224)]} 0 4
[junit4:junit4]   2> 567365 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[8 (1437746168128864256)]} 0 1
[junit4:junit4]   2> 567373 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1437746168134107136)]} 0 1
[junit4:junit4]   2> 567373 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1437746168134107136)]} 0 4
[junit4:junit4]   2> 567374 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[8]} 0 7
[junit4:junit4]   2> 567378 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[9 (1437746168142495744)]} 0 1
[junit4:junit4]   2> 567384 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1437746168145641472)]} 0 1
[junit4:junit4]   2> 567384 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[9 (1437746168145641472)]} 0 4
[junit4:junit4]   2> 567388 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[10 (1437746168152981504)]} 0 1
[junit4:junit4]   2> 567396 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1437746168158224384)]} 0 1
[junit4:junit4]   2> 567397 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1437746168158224384)]} 0 4
[junit4:junit4]   2> 567397 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[10]} 0 7
[junit4:junit4]   2> 567401 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[11 (1437746168166612992)]} 0 1
[junit4:junit4]   2> 567409 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1437746168171855872)]} 0 1
[junit4:junit4]   2> 567409 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1437746168171855872)]} 0 4
[junit4:junit4]   2> 567410 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[11]} 0 7
[junit4:junit4]   2> 567414 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[12 (1437746168180244480)]} 0 1
[junit4:junit4]   2> 567423 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1437746168186535936)]} 0 1
[junit4:junit4]   2> 567423 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1437746168186535936)]} 0 4
[junit4:junit4]   2> 567424 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[12]} 0 7
[junit4:junit4]   2> 567428 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[13 (1437746168194924544)]} 0 1
[junit4:junit4]   2> 567434 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1437746168198070272)]} 0 1
[junit4:junit4]   2> 567434 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[13 (1437746168198070272)]} 0 4
[junit4:junit4]   2> 567438 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[14 (1437746168205410304)]} 0 1
[junit4:junit4]   2> 567445 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1437746168209604608)]} 0 1
[junit4:junit4]   2> 567445 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[14 (1437746168209604608)]} 0 4
[junit4:junit4]   2> 567449 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[15 (1437746168216944640)]} 0 1
[junit4:junit4]   2> 567456 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1437746168221138944)]} 0 1
[junit4:junit4]   2> 567456 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[15 (1437746168221138944)]} 0 4
[junit4:junit4]   2> 567460 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[16 (1437746168228478976)]} 0 1
[junit4:junit4]   2> 567467 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1437746168232673280)]} 0 1
[junit4:junit4]   2> 567467 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[16 (1437746168232673280)]} 0 4
[junit4:junit4]   2> 567471 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[17 (1437746168240013312)]} 0 1
[junit4:junit4]   2> 567479 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1437746168245256192)]} 0 1
[junit4:junit4]   2> 567479 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1437746168245256192)]} 0 4
[junit4:junit4]   2> 567480 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[17]} 0 7
[junit4:junit4]   2> 567484 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[18 (1437746168253644800)]} 0 1
[junit4:junit4]   2> 567493 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1437746168259936256)]} 0 1
[junit4:junit4]   2> 567493 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[18 (1437746168259936256)]} 0 4
[junit4:junit4]   2> 567494 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[18]} 0 7
[junit4:junit4]   2> 567498 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[19 (1437746168268324864)]} 0 1
[junit4:junit4]   2> 567504 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1437746168271470592)]} 0 1
[junit4:junit4]   2> 567504 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[19 (1437746168271470592)]} 0 4
[junit4:junit4]   2> 567508 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[20 (1437746168278810624)]} 0 1
[junit4:junit4]   2> 567516 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1437746168284053504)]} 0 1
[junit4:junit4]   2> 567516 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[20 (1437746168284053504)]} 0 4
[junit4:junit4]   2> 567517 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[20]} 0 7
[junit4:junit4]   2> 567521 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[21 (1437746168292442112)]} 0 1
[junit4:junit4]   2> 567530 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1437746168298733568)]} 0 1
[junit4:junit4]   2> 567530 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1437746168298733568)]} 0 4
[junit4:junit4]   2> 567531 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[21]} 0 7
[junit4:junit4]   2> 567535 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[22 (1437746168307122176)]} 0 1
[junit4:junit4]   2> 567542 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1437746168311316480)]} 0 1
[junit4:junit4]   2> 567542 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[22 (1437746168311316480)]} 0 4
[junit4:junit4]   2> 567546 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[23 (1437746168318656512)]} 0 1
[junit4:junit4]   2> 567552 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1437746168321802240)]} 0 1
[junit4:junit4]   2> 567552 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[23 (1437746168321802240)]} 0 4
[junit4:junit4]   2> 567556 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[24 (1437746168329142272)]} 0 1
[junit4:junit4]   2> 567563 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1437746168333336576)]} 0 1
[junit4:junit4]   2> 567563 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[24 (1437746168333336576)]} 0 4
[junit4:junit4]   2> 567567 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[25 (1437746168340676608)]} 0 1
[junit4:junit4]   2> 567575 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1437746168345919488)]} 0 1
[junit4:junit4]   2> 567575 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[25 (1437746168345919488)]} 0 4
[junit4:junit4]   2> 567576 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[25]} 0 7
[junit4:junit4]   2> 567580 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[26 (1437746168354308096)]} 0 1
[junit4:junit4]   2> 567589 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1437746168360599552)]} 0 1
[junit4:junit4]   2> 567589 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[26 (1437746168360599552)]} 0 4
[junit4:junit4]   2> 567590 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[26]} 0 7
[junit4:junit4]   2> 567594 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[27 (1437746168368988160)]} 0 1
[junit4:junit4]   2> 567603 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1437746168375279616)]} 0 1
[junit4:junit4]   2> 567603 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[27 (1437746168375279616)]} 0 4
[junit4:junit4]   2> 567604 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[27]} 0 7
[junit4:junit4]   2> 567608 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[28 (1437746168383668224)]} 0 1
[junit4:junit4]   2> 567616 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1437746168388911104)]} 0 1
[junit4:junit4]   2> 567616 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[28 (1437746168388911104)]} 0 4
[junit4:junit4]   2> 567617 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[28]} 0 7
[junit4:junit4]   2> 567621 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[29 (1437746168397299712)]} 0 1
[junit4:junit4]   2> 567627 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1437746168400445440)]} 0 1
[junit4:junit4]   2> 567627 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[29 (1437746168400445440)]} 0 4
[junit4:junit4]   2> 567631 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[30 (1437746168407785472)]} 0 1
[junit4:junit4]   2> 567638 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1437746168411979776)]} 0 1
[junit4:junit4]   2> 567638 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[30 (1437746168411979776)]} 0 4
[junit4:junit4]   2> 567642 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[31 (1437746168419319808)]} 0 1
[junit4:junit4]   2> 567650 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1437746168424562688)]} 0 1
[junit4:junit4]   2> 567650 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[31 (1437746168424562688)]} 0 4
[junit4:junit4]   2> 567651 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[31]} 0 7
[junit4:junit4]   2> 567655 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[32 (1437746168432951296)]} 0 1
[junit4:junit4]   2> 567661 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1437746168436097024)]} 0 1
[junit4:junit4]   2> 567661 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[32 (1437746168436097024)]} 0 4
[junit4:junit4]   2> 567665 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[33 (1437746168443437056)]} 0 1
[junit4:junit4]   2> 567674 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1437746168449728512)]} 0 1
[junit4:junit4]   2> 567674 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[33 (1437746168449728512)]} 0 4
[junit4:junit4]   2> 567675 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[33]} 0 7
[junit4:junit4]   2> 567679 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[34 (1437746168458117120)]} 0 1
[junit4:junit4]   2> 567687 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1437746168463360000)]} 0 1
[junit4:junit4]   2> 567687 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[34 (1437746168463360000)]} 0 4
[junit4:junit4]   2> 567688 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[34]} 0 7
[junit4:junit4]   2> 567692 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[35 (1437746168471748608)]} 0 1
[junit4:junit4]   2> 567698 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1437746168474894336)]} 0 1
[junit4:junit4]   2> 567698 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[35 (1437746168474894336)]} 0 4
[junit4:junit4]   2> 567702 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[36 (1437746168482234368)]} 0 1
[junit4:junit4]   2> 567708 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1437746168485380096)]} 0 1
[junit4:junit4]   2> 567708 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[36 (1437746168485380096)]} 0 4
[junit4:junit4]   2> 567712 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[37 (1437746168492720128)]} 0 1
[junit4:junit4]   2> 567721 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1437746168499011584)]} 0 1
[junit4:junit4]   2> 567721 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[37 (1437746168499011584)]} 0 4
[junit4:junit4]   2> 567722 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[37]} 0 7
[junit4:junit4]   2> 567726 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[38 (1437746168507400192)]} 0 1
[junit4:junit4]   2> 567734 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1437746168512643072)]} 0 1
[junit4:junit4]   2> 567734 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[38 (1437746168512643072)]} 0 4
[junit4:junit4]   2> 567735 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[38]} 0 7
[junit4:junit4]   2> 567739 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[39 (1437746168521031680)]} 0 1
[junit4:junit4]   2> 567746 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1437746168525225984)]} 0 1
[junit4:junit4]   2> 567746 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[39 (1437746168525225984)]} 0 4
[junit4:junit4]   2> 567750 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[40 (1437746168532566016)]} 0 1
[junit4:junit4]   2> 567756 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1437746168535711744)]} 0 1
[junit4:junit4]   2> 567756 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[40 (1437746168535711744)]} 0 4
[junit4:junit4]   2> 567760 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[41 (1437746168543051776)]} 0 1
[junit4:junit4]   2> 567769 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1437746168549343232)]} 0 1
[junit4:junit4]   2> 567769 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[41 (1437746168549343232)]} 0 4
[junit4:junit4]   2> 567770 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[41]} 0 7
[junit4:junit4]   2> 567774 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[42 (1437746168557731840)]} 0 1
[junit4:junit4]   2> 567782 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1437746168562974720)]} 0 1
[junit4:junit4]   2> 567782 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[42 (1437746168562974720)]} 0 4
[junit4:junit4]   2> 567783 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[42]} 0 7
[junit4:junit4]   2> 567787 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[43 (1437746168571363328)]} 0 1
[junit4:junit4]   2> 567796 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1437746168577654784)]} 0 1
[junit4:junit4]   2> 567796 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[43 (1437746168577654784)]} 0 4
[junit4:junit4]   2> 567797 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[43]} 0 7
[junit4:junit4]   2> 567801 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[44 (1437746168586043392)]} 0 1
[junit4:junit4]   2> 567810 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1437746168592334848)]} 0 1
[junit4:junit4]   2> 567810 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[44 (1437746168592334848)]} 0 4
[junit4:junit4]   2> 567811 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[44]} 0 7
[junit4:junit4]   2> 567815 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[45 (1437746168600723456)]} 0 1
[junit4:junit4]   2> 567821 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1437746168603869184)]} 0 1
[junit4:junit4]   2> 567821 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[45 (1437746168603869184)]} 0 4
[junit4:junit4]   2> 567825 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[46 (1437746168611209216)]} 0 1
[junit4:junit4]   2> 567831 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1437746168614354944)]} 0 1
[junit4:junit4]   2> 567831 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[46 (1437746168614354944)]} 0 4
[junit4:junit4]   2> 567835 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[47 (1437746168621694976)]} 0 1
[junit4:junit4]   2> 567842 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1437746168625889280)]} 0 1
[junit4:junit4]   2> 567843 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[47 (1437746168625889280)]} 0 5
[junit4:junit4]   2> 567847 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[48 (1437746168634277888)]} 0 1
[junit4:junit4]   2> 567854 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1437746168638472192)]} 0 1
[junit4:junit4]   2> 567854 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[48 (1437746168638472192)]} 0 4
[junit4:junit4]   2> 567858 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[49 (1437746168645812224)]} 0 1
[junit4:junit4]   2> 567864 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1437746168648957952)]} 0 1
[junit4:junit4]   2> 567864 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[49 (1437746168648957952)]} 0 4
[junit4:junit4]   2> 567868 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[50 (1437746168656297984)]} 0 1
[junit4:junit4]   2> 567874 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1437746168659443712)]} 0 1
[junit4:junit4]   2> 567875 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[50 (1437746168659443712)]} 0 5
[junit4:junit4]   2> 567879 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[51 (1437746168667832320)]} 0 1
[junit4:junit4]   2> 567885 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1437746168670978048)]} 0 1
[junit4:junit4]   2> 567885 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[51 (1437746168670978048)]} 0 4
[junit4:junit4]   2> 567889 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[52 (1437746168678318080)]} 0 1
[junit4:junit4]   2> 567895 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1437746168681463808)]} 0 1
[junit4:junit4]   2> 567895 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[52 (1437746168681463808)]} 0 4
[junit4:junit4]   2> 567899 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[53 (1437746168688803840)]} 0 1
[junit4:junit4]   2> 567905 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1437746168691949568)]} 0 1
[junit4:junit4]   2> 567906 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[53 (1437746168691949568)]} 0 5
[junit4:junit4]   2> 567910 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[54 (1437746168700338176)]} 0 1
[junit4:junit4]   2> 567919 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1437746168706629632)]} 0 1
[junit4:junit4]   2> 567920 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[54 (1437746168706629632)]} 0 5
[junit4:junit4]   2> 567921 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[54]} 0 8
[junit4:junit4]   2> 567924 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[55 (1437746168716066816)]} 0 1
[junit4:junit4]   2> 567931 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1437746168719212544)]} 0 1
[junit4:junit4]   2> 567932 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[55 (1437746168719212544)]} 0 5
[junit4:junit4]   2> 567936 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[56 (1437746168727601152)]} 0 1
[junit4:junit4]   2> 567946 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1437746168734941184)]} 0 1
[junit4:junit4]   2> 567947 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[56 (1437746168734941184)]} 0 5
[junit4:junit4]   2> 567947 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[56]} 0 8
[junit4:junit4]   2> 567951 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[57 (1437746168743329792)]} 0 1
[junit4:junit4]   2> 567960 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1437746168749621248)]} 0 1
[junit4:junit4]   2> 567961 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[57 (1437746168749621248)]} 0 5
[junit4:junit4]   2> 567962 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[57]} 0 8
[junit4:junit4]   2> 567966 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[58 (1437746168759058432)]} 0 1
[junit4:junit4]   2> 567975 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1437746168765349888)]} 0 1
[junit4:junit4]   2> 567976 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[58 (1437746168765349888)]} 0 5
[junit4:junit4]   2> 567977 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[58]} 0 8
[junit4:junit4]   2> 567981 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[59 (1437746168774787072)]} 0 1
[junit4:junit4]   2> 567988 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1437746168778981376)]} 0 1
[junit4:junit4]   2> 567989 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[59 (1437746168778981376)]} 0 5
[junit4:junit4]   2> 567993 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[60 (1437746168787369984)]} 0 1
[junit4:junit4]   2> 568002 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[60 (1437746168793661440)]} 0 1
[junit4:junit4]   2> 568003 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[60 (1437746168793661440)]} 0 5
[junit4:junit4]   2> 568004 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[60]} 0 8
[junit4:junit4]   2> 568008 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[61 (1437746168803098624)]} 0 1
[junit4:junit4]   2> 568018 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[61 (1437746168810438656)]} 0 1
[junit4:junit4]   2> 568018 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[61 (1437746168810438656)]} 0 4
[junit4:junit4]   2> 568019 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[61]} 0 8
[junit4:junit4]   2> 568023 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[62 (1437746168818827264)]} 0 1
[junit4:junit4]   2> 568032 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[62 (1437746168825118720)]} 0 1
[junit4:junit4]   2> 568032 T914 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[62 (1437746168825118720)]} 0 4
[junit4:junit4]   2> 568033 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[62]} 0 7
[junit4:junit4]   2> 568037 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[63 (1437746168833507328)]} 0 1
[junit4:junit4]   2> 568044 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[63 (1437746168837701632)]} 0 1
[junit4:junit4]   2> 568045 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[63 (1437746168837701632)]} 0 5
[junit4:junit4]   2> 568049 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[64 (1437746168846090240)]} 0 1
[junit4:junit4]   2> 568056 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[64 (1437746168850284544)]} 0 1
[junit4:junit4]   2> 568057 T898 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[64 (1437746168850284544)]} 0 5
[junit4:junit4]   2> 568061 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[65 (1437746168858673152)]} 0 1
[junit4:junit4]   2> 568071 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[65 (1437746168866013184)]} 0 1
[junit4:junit4]   2> 568072 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[65 (1437746168866013184)]} 0 5
[junit4:junit4]   2> 568072 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[65]} 0 8
[junit4:junit4]   2> 568076 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[66 (1437746168874401792)]} 0 1
[junit4:junit4]   2> 568086 T926 C166 P62951 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:41428/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[66 (1437746168881741824)]} 0 1
[junit4:junit4]   2> 568087 T896 C164 P41428 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[66 (1437746168881741824)]} 0 5
[junit4:junit4]   2> 568088 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[66]} 0 9
[junit4:junit4]   2> 568091 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[67 (1437746168891179008)]} 0 1
[junit4:junit4]   2> 568098 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[67 (1437746168894324736)]} 0 1
[junit4:junit4]   2> 568099 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[67 (1437746168894324736)]} 0 5
[junit4:junit4]   2> 568103 T872 C162 P27612 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[68 (1437746168902713344)]} 0 1
[junit4:junit4]   2> 568110 T941 C165 P32634 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={distrib.from=http://127.0.0.1:14096/y_q/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[68 (1437746168906907648)]} 0 1
[junit4:junit4]   2> 568111 T912 C163 P14096 oasup.LogUpdateProcessor.finish [collection1] webapp=/y_q path=/update params={wt=javabin&version=2} {add=[68 (1437746168906907648)]} 0 5
[junit4:junit4]   2> 568115 T872 C162 P27612 oasup.L

[...truncated too long message...]

  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4]   2> 773186 T1013 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:62951_y_q_collection1_shard1_0_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1_0",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":null,
[junit4:junit4]   2> 	  "core":"collection1_shard1_0_replica1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4]   2> 773197 T1013 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:62951_y_q_collection1_shard1_1_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1_1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":null,
[junit4:junit4]   2> 	  "core":"collection1_shard1_1_replica1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:62951_y_q",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:62951/y_q"}
[junit4:junit4]   2> 773209 T860 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 32634
[junit4:junit4]   2> 773209 T860 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1242169106
[junit4:junit4]   2> 773222 T951 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4]   2> 773223 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 774222 T860 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 774222 T860 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 774225 T860 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5b967893
[junit4:junit4]   2> 774227 T860 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=4,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=861,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 774228 T860 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 774228 T860 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 774229 T860 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 774231 T860 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 774232 T860 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4]   2> 774232 T860 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4;done=false>>]
[junit4:junit4]   2> 774233 T860 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4
[junit4:junit4]   2> 774233 T860 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index;done=false>>]
[junit4:junit4]   2> 774234 T860 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910/jetty4/index
[junit4:junit4]   2> 774235 T1013 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89859133664067596-127.0.0.1:32634_y_q-n_0000000004) am no longer a leader.
[junit4:junit4]   2> 774236 T951 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 774236 T951 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 774237 T860 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/y_q,null}
[junit4:junit4]   2> 774237 T951 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 774309 T860 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> 774311 T860 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:55935 55935
[junit4:junit4]   2> 774454 T934 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 774583 T934 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 774583 T934 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 774584 T860 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 27612
[junit4:junit4]   2> !!!! WARNING: best effort to remove /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1371141565910 FAILED !!!!!
[junit4:junit4]   2> 774585 T860 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> 774586 T860 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:55935 55935
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeyShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=94C1FB14CE220E9A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=be -Dtests.timezone=America/Kralendijk -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR    236s J1 | ChaosMonkeyShardSplitTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:41428/y_q returned non ok status:500, message:Server Error
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([94C1FB14CE220E9A:1527750CB97D6EA6]:0)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4]    > 	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
[junit4:junit4]    > 	at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 774744 T860 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 235538 T859 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4]   2> NOTE: test params are: codec=Appending, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=be, timezone=America/Kralendijk
[junit4:junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=1,free=160813512,total=387973120
[junit4:junit4]   2> NOTE: All tests run in this JVM: [ShowFileRequestHandlerTest, StandardRequestHandlerTest, SuggesterWFSTTest, FieldMutatingUpdateProcessorTest, TestMaxScoreQueryParser, ShardRoutingCustomTest, TestSolrXMLSerializer, CollectionsAPIDistributedZkTest, ShardSplitTest, ShardRoutingTest, SolrCoreCheckLockOnStartupTest, TestBadConfig, DOMUtilTest, DocValuesMultiTest, PrimUtilsTest, TestLRUCache, SuggesterTSTTest, TestDynamicFieldResource, IndexSchemaRuntimeFieldTest, TestHashPartitioner, TestFieldTypeCollectionResource, CacheHeaderTest, CSVRequestHandlerTest, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J1 in 236.33s, 1 test, 1 error <<< FAILURES!

[...truncated 731 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:391: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:371: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:181: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:437: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:887: There were test failures: 297 suites, 1246 tests, 1 error, 13 ignored (7 assumptions)

Total time: 51 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message