lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Policeman Jenkins Server <jenk...@thetaphi.de>
Subject [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_60-ea-b13) - Build # 10061 - Failure!
Date Sat, 12 Apr 2014 15:39:10 GMT
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10061/
Java: 32bit/jdk1.7.0_60-ea-b13 -server -XX:+UseG1GC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.ShardSplitTest.testDistribSearch

Error Message:
collection already exists: splitByRouteKeyTest

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: collection already exists: splitByRouteKeyTest
	at __randomizedtesting.SeedInfo.seed([58C47A91C07A691B:D922F489B7250927]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:340)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:301)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:636)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1648)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1603)
	at org.apache.solr.cloud.ShardSplitTest.splitByRouteKeyTest(ShardSplitTest.java:337)
	at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:103)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
	at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 11571 lines...]
   [junit4] Suite: org.apache.solr.cloud.ShardSplitTest
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/init-core-data-001
   [junit4]   2> 1185570 T6018 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 1185573 T6018 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 1185573 T6018 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1185574 T6019 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1185674 T6018 oasc.ZkTestServer.run start zk server on port:47253
   [junit4]   2> 1185675 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1207672 T6022 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 21995ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
   [junit4]   2> 1207674 T6020 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14556933c8f0000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 1207675 T6020 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14556933c8f0001, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 1209608 T6025 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c8743c name:ZooKeeperConnection Watcher:127.0.0.1:47253 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1209608 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1209609 T6018 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1209615 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1209616 T6027 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a524db name:ZooKeeperConnection Watcher:127.0.0.1:47253/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1209616 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1209617 T6018 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1209621 T6018 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1209623 T6018 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1209625 T6018 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1209627 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1209627 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1209631 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1209631 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1209635 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1209635 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1209638 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1209638 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1209640 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1209641 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1209642 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1209643 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 1209645 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1209645 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1209647 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1209647 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1209649 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1209649 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1209651 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1209651 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1209654 T6018 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1209654 T6018 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1209888 T6018 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1209891 T6018 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:56343
   [junit4]   2> 1209892 T6018 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1209892 T6018 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1209892 T6018 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-002
   [junit4]   2> 1209893 T6018 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-002/'
   [junit4]   2> 1209909 T6018 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-002/solr.xml
   [junit4]   2> 1209935 T6018 oasc.CoreContainer.<init> New CoreContainer 14431251
   [junit4]   2> 1209936 T6018 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-002/]
   [junit4]   2> 1209936 T6018 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1209937 T6018 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1209937 T6018 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1209938 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1209938 T6018 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1209938 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1209939 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1209939 T6018 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1209939 T6018 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1209940 T6018 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1209940 T6018 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1209941 T6018 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1209941 T6018 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47253/solr
   [junit4]   2> 1209941 T6018 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1209942 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1209943 T6038 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@df085b name:ZooKeeperConnection Watcher:127.0.0.1:47253 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1209944 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1209945 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1209946 T6040 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f7e2e4 name:ZooKeeperConnection Watcher:127.0.0.1:47253/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1209946 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1209947 T6018 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1209949 T6018 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 1209951 T6018 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 1209953 T6018 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 1209955 T6018 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 1209957 T6018 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 1209958 T6018 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56343_
   [junit4]   2> 1209959 T6018 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56343_
   [junit4]   2> 1209960 T6018 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 1209963 T6018 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 1209968 T6018 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56343_
   [junit4]   2> 1209969 T6018 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1209971 T6018 oasc.Overseer.start Overseer (id=91574557807149061-127.0.0.1:56343_-n_0000000000) starting
   [junit4]   2> 1209973 T6018 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 1209979 T6042 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1209980 T6018 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1209980 T6042 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:56343_
   [junit4]   2> 1209981 T6018 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1209982 T6018 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1209984 T6041 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1209987 T6043 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 1209987 T6043 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1209988 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1209988 T6043 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1209989 T6041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1209990 T6041 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56343",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:56343_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1209990 T6041 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 1209990 T6041 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1209992 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1210000 T6040 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1210989 T6043 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1210989 T6043 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 1210990 T6043 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1210990 T6043 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 1210991 T6043 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1210991 T6043 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-002/collection1/'
   [junit4]   2> 1210992 T6043 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 1210992 T6043 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 1210992 T6043 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 1211021 T6043 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1211046 T6043 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1211047 T6043 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1211054 T6043 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1211273 T6043 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1211275 T6043 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1211276 T6043 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1211279 T6043 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1211292 T6043 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1211293 T6043 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1211293 T6043 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-002/collection1/, dataDir=./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/control/data/
   [junit4]   2> 1211293 T6043 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@16e125
   [junit4]   2> 1211294 T6043 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/control/data
   [junit4]   2> 1211294 T6043 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/control/data/index/
   [junit4]   2> 1211294 T6043 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1211294 T6043 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/control/data/index
   [junit4]   2> 1211295 T6043 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=37.484375, floorSegmentMB=0.4921875, forceMergeDeletesPctAllowed=3.4510966396004648, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1211295 T6043 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@88f7e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1be29d7),segFN=segments_1,generation=1}
   [junit4]   2> 1211296 T6043 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1211299 T6043 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1211299 T6043 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1211299 T6043 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1211299 T6043 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1211300 T6043 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1211300 T6043 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1211300 T6043 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1211301 T6043 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1211301 T6043 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1211301 T6043 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1211302 T6043 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1211302 T6043 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1211302 T6043 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1211302 T6043 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1211303 T6043 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1211303 T6043 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1211304 T6043 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 1211306 T6043 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1211308 T6043 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1211308 T6043 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1211309 T6043 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=16.833984375, floorSegmentMB=0.8369140625, forceMergeDeletesPctAllowed=29.062808810988166, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1211309 T6043 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@88f7e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1be29d7),segFN=segments_1,generation=1}
   [junit4]   2> 1211309 T6043 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1211310 T6043 oass.SolrIndexSearcher.<init> Opening Searcher@1f18f5c[collection1] main
   [junit4]   2> 1211310 T6043 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 1211311 T6043 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1211311 T6043 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1211311 T6043 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1211311 T6043 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1211312 T6043 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1211312 T6043 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1211312 T6043 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1211312 T6043 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 1211313 T6043 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1211314 T6044 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f18f5c[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1211316 T6043 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1211316 T6047 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56343 collection:control_collection shard:shard1
   [junit4]   2> 1211317 T6018 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 1211317 T6018 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1211317 T6047 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1211318 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1211319 T6049 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13f384 name:ZooKeeperConnection Watcher:127.0.0.1:47253/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1211320 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1211321 T6018 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1211322 T6047 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1211322 T6018 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1211323 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1211324 T6047 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1211324 T6047 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C1473 name=collection1 org.apache.solr.core.SolrCore@bea6bf url=http://127.0.0.1:56343/collection1 node=127.0.0.1:56343_ C1473_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:56343, core=collection1, node_name=127.0.0.1:56343_}
   [junit4]   2> 1211324 T6047 C1473 P56343 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56343/collection1/
   [junit4]   2> 1211324 T6041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1211324 T6047 C1473 P56343 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1211325 T6047 C1473 P56343 oasc.SyncStrategy.syncToMe http://127.0.0.1:56343/collection1/ has no replicas
   [junit4]   2> 1211325 T6047 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56343/collection1/ shard1
   [junit4]   2> 1211325 T6047 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1211326 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1211328 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1211330 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1211432 T6049 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1211432 T6040 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1211479 T6047 oasc.ZkController.register We are http://127.0.0.1:56343/collection1/ and leader is http://127.0.0.1:56343/collection1/
   [junit4]   2> 1211479 T6047 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56343
   [junit4]   2> 1211479 T6047 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1211480 T6047 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 1211480 T6047 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1211480 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1211481 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1211481 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1211481 T6047 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1211483 T6041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1211483 T6041 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56343",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:56343_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 1211484 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1211550 T6018 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 1211551 T6018 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1211552 T6018 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41815
   [junit4]   2> 1211553 T6018 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1211553 T6018 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1211554 T6018 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-003
   [junit4]   2> 1211554 T6018 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-003/'
   [junit4]   2> 1211569 T6018 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-003/solr.xml
   [junit4]   2> 1211593 T6040 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1211593 T6049 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1211597 T6018 oasc.CoreContainer.<init> New CoreContainer 30703554
   [junit4]   2> 1211597 T6018 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-003/]
   [junit4]   2> 1211598 T6018 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1211598 T6018 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1211599 T6018 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1211599 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1211599 T6018 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1211600 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1211600 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1211600 T6018 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1211601 T6018 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1211601 T6018 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1211602 T6018 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1211602 T6018 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1211602 T6018 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47253/solr
   [junit4]   2> 1211603 T6018 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1211604 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1211605 T6060 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fc8a7 name:ZooKeeperConnection Watcher:127.0.0.1:47253 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1211605 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1211607 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1211607 T6062 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1994aef name:ZooKeeperConnection Watcher:127.0.0.1:47253/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1211608 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1211614 T6018 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1212616 T6018 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41815_
   [junit4]   2> 1212618 T6018 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41815_
   [junit4]   2> 1212619 T6040 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1212620 T6049 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1212620 T6062 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1212624 T6063 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 1212624 T6063 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1212625 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1212625 T6063 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1212625 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1212626 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1212626 T6041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1212627 T6041 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:41815",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:41815_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1212627 T6041 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 1212627 T6041 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1212629 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1212730 T6062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1212730 T6049 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1212730 T6040 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1213626 T6063 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1213626 T6063 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1213627 T6063 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1213627 T6063 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1213628 T6063 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1213628 T6063 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-003/collection1/'
   [junit4]   2> 1213628 T6063 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 1213629 T6063 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 1213629 T6063 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 1213659 T6063 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1213684 T6063 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1213685 T6063 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1213693 T6063 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1213894 T6063 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1213896 T6063 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1213897 T6063 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1213900 T6063 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1213913 T6063 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1213914 T6063 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1213914 T6063 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-003/collection1/, dataDir=./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/jetty1/
   [junit4]   2> 1213914 T6063 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@16e125
   [junit4]   2> 1213915 T6063 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/jetty1
   [junit4]   2> 1213915 T6063 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/jetty1/index/
   [junit4]   2> 1213915 T6063 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 1213916 T6063 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/jetty1/index
   [junit4]   2> 1213916 T6063 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=37.484375, floorSegmentMB=0.4921875, forceMergeDeletesPctAllowed=3.4510966396004648, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1213917 T6063 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@9c5723 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@378edf),segFN=segments_1,generation=1}
   [junit4]   2> 1213917 T6063 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1213920 T6063 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1213921 T6063 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1213921 T6063 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1213921 T6063 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1213922 T6063 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1213922 T6063 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1213922 T6063 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1213922 T6063 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1213923 T6063 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1213923 T6063 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1213924 T6063 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1213924 T6063 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1213924 T6063 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1213925 T6063 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1213925 T6063 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1213925 T6063 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1213926 T6063 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 1213928 T6063 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1213930 T6063 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1213930 T6063 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1213931 T6063 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=16.833984375, floorSegmentMB=0.8369140625, forceMergeDeletesPctAllowed=29.062808810988166, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1213931 T6063 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@9c5723 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@378edf),segFN=segments_1,generation=1}
   [junit4]   2> 1213932 T6063 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1213932 T6063 oass.SolrIndexSearcher.<init> Opening Searcher@c27337[collection1] main
   [junit4]   2> 1213932 T6063 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1213933 T6063 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1213933 T6063 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1213934 T6063 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1213934 T6063 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1213934 T6063 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1213934 T6063 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1213935 T6063 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1213935 T6063 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 1213935 T6063 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1213938 T6064 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c27337[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1213939 T6063 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1213940 T6067 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41815 collection:collection1 shard:shard1
   [junit4]   2> 1213940 T6018 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 1213941 T6018 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1213945 T6067 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1213949 T6067 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1213951 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1213951 T6067 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1213951 T6067 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 1213951 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> ASYNC  NEW_CORE C1474 name=collection1 org.apache.solr.core.SolrCore@1dca45c url=http://127.0.0.1:41815/collection1 node=127.0.0.1:41815_ C1474_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:41815, core=collection1, node_name=127.0.0.1:41815_}
   [junit4]   2> 1213951 T6067 C1474 P41815 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41815/collection1/
   [junit4]   2> 1213952 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1213952 T6067 C1474 P41815 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1213952 T6067 C1474 P41815 oasc.SyncStrategy.syncToMe http://127.0.0.1:41815/collection1/ has no replicas
   [junit4]   2> 1213953 T6041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1213953 T6067 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41815/collection1/ shard1
   [junit4]   2> 1213953 T6067 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1213955 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1213957 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1213959 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1214061 T6062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1214061 T6040 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1214061 T6049 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1214108 T6067 oasc.ZkController.register We are http://127.0.0.1:41815/collection1/ and leader is http://127.0.0.1:41815/collection1/
   [junit4]   2> 1214108 T6067 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41815
   [junit4]   2> 1214109 T6067 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1214109 T6067 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 1214109 T6067 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1214110 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1214110 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1214110 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1214111 T6067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1214111 T6041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1214112 T6041 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:41815",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:41815_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 1214113 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1214206 T6018 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 1214207 T6018 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1214209 T6018 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44081
   [junit4]   2> 1214209 T6018 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1214210 T6018 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1214210 T6018 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-004
   [junit4]   2> 1214210 T6018 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-004/'
   [junit4]   2> 1214219 T6040 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1214219 T6062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1214219 T6049 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1214227 T6018 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-004/solr.xml
   [junit4]   2> 1214253 T6018 oasc.CoreContainer.<init> New CoreContainer 28368140
   [junit4]   2> 1214253 T6018 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-004/]
   [junit4]   2> 1214254 T6018 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1214255 T6018 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1214255 T6018 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1214255 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1214256 T6018 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1214256 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1214256 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1214257 T6018 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1214257 T6018 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1214258 T6018 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1214258 T6018 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1214258 T6018 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1214259 T6018 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47253/solr
   [junit4]   2> 1214259 T6018 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1214260 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1214261 T6078 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@807a2a name:ZooKeeperConnection Watcher:127.0.0.1:47253 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1214262 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1214263 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1214264 T6080 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d9af23 name:ZooKeeperConnection Watcher:127.0.0.1:47253/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1214264 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1214268 T6018 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1215270 T6018 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44081_
   [junit4]   2> 1215271 T6018 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44081_
   [junit4]   2> 1215273 T6062 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1215273 T6040 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1215274 T6080 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1215274 T6049 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1215279 T6081 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 1215280 T6081 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1215281 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1215281 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1215281 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1215281 T6081 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1215282 T6041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1215283 T6041 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44081",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:44081_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1215283 T6041 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 1215283 T6041 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 1215285 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1215387 T6062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1215387 T6080 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1215387 T6040 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1215387 T6049 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1216282 T6081 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1216282 T6081 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1216283 T6081 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1216283 T6081 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1216284 T6081 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1216284 T6081 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-004/collection1/'
   [junit4]   2> 1216284 T6081 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 1216285 T6081 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 1216285 T6081 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 1216315 T6081 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1216340 T6081 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1216341 T6081 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1216347 T6081 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1216538 T6081 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1216540 T6081 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1216541 T6081 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1216544 T6081 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1216557 T6081 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1216557 T6081 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1216558 T6081 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-004/collection1/, dataDir=./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/jetty2/
   [junit4]   2> 1216558 T6081 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@16e125
   [junit4]   2> 1216558 T6081 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/jetty2
   [junit4]   2> 1216559 T6081 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/jetty2/index/
   [junit4]   2> 1216559 T6081 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 1216559 T6081 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-001/jetty2/index
   [junit4]   2> 1216559 T6081 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=37.484375, floorSegmentMB=0.4921875, forceMergeDeletesPctAllowed=3.4510966396004648, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1216560 T6081 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@b550ab lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39c4ab),segFN=segments_1,generation=1}
   [junit4]   2> 1216560 T6081 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1216564 T6081 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1216564 T6081 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1216565 T6081 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1216565 T6081 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1216565 T6081 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1216565 T6081 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1216566 T6081 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1216566 T6081 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1216566 T6081 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1216567 T6081 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1216567 T6081 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1216567 T6081 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1216567 T6081 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1216568 T6081 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1216568 T6081 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1216569 T6081 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1216569 T6081 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 1216572 T6081 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1216573 T6081 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1216574 T6081 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1216574 T6081 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=16.833984375, floorSegmentMB=0.8369140625, forceMergeDeletesPctAllowed=29.062808810988166, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1216575 T6081 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@b550ab lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@39c4ab),segFN=segments_1,generation=1}
   [junit4]   2> 1216575 T6081 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1216575 T6081 oass.SolrIndexSearcher.<init> Opening Searcher@1b8fbb4[collection1] main
   [junit4]   2> 1216575 T6081 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1216576 T6081 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1216577 T6081 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1216577 T6081 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1216577 T6081 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1216577 T6081 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1216578 T6081 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1216578 T6081 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1216578 T6081 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 1216578 T6081 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1216581 T6082 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b8fbb4[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1216583 T6081 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1216583 T6085 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44081 collection:collection1 shard:shard2
   [junit4]   2> 1216584 T6018 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 1216584 T6018 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1216585 T6085 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 1216589 T6085 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 1216591 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1216591 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1216591 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1216591 T6085 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1216592 T6085 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C1475 name=collection1 org.apache.solr.core.SolrCore@3f9160 url=http://127.0.0.1:44081/collection1 node=127.0.0.1:44081_ C1475_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:44081, core=collection1, node_name=127.0.0.1:44081_}
   [junit4]   2> 1216592 T6085 C1475 P44081 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44081/collection1/
   [junit4]   2> 1216592 T6041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1216592 T6085 C1475 P44081 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1216593 T6085 C1475 P44081 oasc.SyncStrategy.syncToMe http://127.0.0.1:44081/collection1/ has no replicas
   [junit4]   2> 1216593 T6085 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44081/collection1/ shard2
   [junit4]   2> 1216593 T6085 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 1216595 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1216596 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1216598 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1216700 T6062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1216700 T6080 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1216700 T6049 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1216700 T6040 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1216747 T6085 oasc.ZkController.register We are http://127.0.0.1:44081/collection1/ and leader is http://127.0.0.1:44081/collection1/
   [junit4]   2> 1216747 T6085 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44081
   [junit4]   2> 1216747 T6085 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1216748 T6085 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 1216748 T6085 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1216749 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1216749 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1216749 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1216749 T6085 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1216750 T6041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1216750 T6041 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44081",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:44081_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 1216752 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1216814 T6018 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 1216814 T6018 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1216816 T6018 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58241
   [junit4]   2> 1216817 T6018 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1216817 T6018 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1216818 T6018 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-005
   [junit4]   2> 1216818 T6018 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-005/'
   [junit4]   2> 1216834 T6018 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-005/solr.xml
   [junit4]   2> 1216855 T6040 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1216855 T6080 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1216855 T6062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1216855 T6049 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1216860 T6018 oasc.CoreContainer.<init> New CoreContainer 33395665
   [junit4]   2> 1216861 T6018 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.ShardSplitTest-58C47A91C07A691B-001/tempDir-005/]
   [junit4]   2> 1216862 T6018 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1216862 T6018 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1216862 T6018 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1216863 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1216863 T6018 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1216863 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1216864 T6018 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1216864 T6018 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1216864 T6018 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1216865 T6018 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1216865 T6018 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1216866 T6018 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1216866 T6018 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47253/solr
   [junit4]   2> 1216866 T6018 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1216867 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1216869 T6096 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f3a75e name:ZooKeeperConnection Watcher:127.0.0.1:47253 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1216869 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1216871 T6018 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1216872 T6098 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13a66f3 name:ZooKeeperConnection Watcher:127.0.0.1:47253/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1216872 T6018 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1216876 T6018 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1217878 T6018 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58241_
   [junit4]   2> 1217880 T6018 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58241_
   [junit4]   2> 1217881 T6049 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1217881 T6062 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1217881 T6040 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1217881 T6098 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1217882 T6080 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1217886 T6099 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 1217887 T6099 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1217887 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1217888 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1217888 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1217888 T6099 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1217889 T6041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1217889 T6041 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:58241",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:58241_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1217890 T6041 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 1217890 T6041 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1217893 T6040 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1217995 T6040 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 1217995 T6049 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 1217995 T6080 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 1217995 T6098 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 1217995 T6062 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 1218888 T6099 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1218889 T6099 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1218889 T6099 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1218890 T6099 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1218891 T6099 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1218891 T6099 oasc.SolrResourceLoader.<init> new SolrReso

[...truncated too long message...]

     /solr/collections/control_collection/leaders (1)
   [junit4]   1>      /solr/collections/control_collection/leaders/shard1 (0)
   [junit4]   1>      DATA:
   [junit4]   1>          {
   [junit4]   1>            "core":"collection1",
   [junit4]   1>            "base_url":"http://127.0.0.1:56343",
   [junit4]   1>            "node_name":"127.0.0.1:56343_"}
   [junit4]   1>  /zookeeper (1)
   [junit4]   1>  DATA:
   [junit4]   1>      
   [junit4]   1> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=58C47A91C07A691B -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=no_NO_NY -Dtests.timezone=IET -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR    249s J0 | ShardSplitTest.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: collection already exists: splitByRouteKeyTest
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([58C47A91C07A691B:D922F489B7250927]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:554)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:340)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:301)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:636)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1648)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1603)
   [junit4]    > 	at org.apache.solr.cloud.ShardSplitTest.splitByRouteKeyTest(ShardSplitTest.java:337)
   [junit4]    > 	at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:103)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 1435058 T6018 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.ShardSplitTest-58C47A91C07A691B-001
   [junit4]   2> 249491 T6017 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
   [junit4]   2> 1436231 T6118 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1436231 T6270 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:302)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:291)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Lucene46: {n_ti=PostingsFormat(name=Memory doPackFST= false), _version_=PostingsFormat(name=Memory doPackFST= false), shard_s=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=Memory doPackFST= true)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=no_NO_NY, timezone=IET
   [junit4]   2> NOTE: Linux 3.8.0-38-generic i386/Oracle Corporation 1.7.0_60-ea (32-bit)/cpus=8,threads=1,free=145438216,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestDocSet, TestCloudManagedSchemaAddField, OverseerTest, TestConfigSets, TestSystemIdResolver, TestStressVersions, BasicDistributedZkTest, ScriptEngineTest, SliceStateTest, SOLR749Test, DistributedQueryElevationComponentTest, TestGroupingSearch, TestNonNRTOpen, TestRestManager, SliceStateUpdateTest, AutoCommitTest, CoreAdminHandlerTest, SearchHandlerTest, URLClassifyProcessorTest, TestFastLRUCache, TestBM25SimilarityFactory, TestFieldTypeCollectionResource, MigrateRouteKeyTest, NoFacetTest, TestOverriddenPrefixQueryForCustomFieldType, TestDistribDocBasedVersion, AbstractAnalyticsFacetTest, DeleteInactiveReplicaTest, TestRemoteStreaming, CircularListTest, RAMDirectoryFactoryTest, FastVectorHighlighterTest, TestSolrQueryParserDefaultOperatorResource, BJQParserTest, SolrIndexSplitterTest, MoreLikeThisHandlerTest, TestInitQParser, TestSolrIndexConfig, DocValuesMissingTest, DirectUpdateHandlerOptimizeTest, BlockDirectoryTest, TestReloadAndDeleteDocs, ChaosMonkeyNothingIsSafeTest, TestCoreContainer, TestTrie, PolyFieldTest, TestCloudManagedSchema, ClusterStateTest, TermVectorComponentTest, SampleTest, DocumentBuilderTest, LukeRequestHandlerTest, AddBlockUpdateTest, TestRandomMergePolicy, TestDynamicFieldCollectionResource, OverseerCollectionProcessorTest, TestFileDictionaryLookup, TestSuggestSpellingConverter, DistributedExpandComponentTest, TestExtendedDismaxParser, TestManagedResourceStorage, UUIDFieldTest, TestNRTOpen, DistributedQueryComponentOptimizationTest, TestCollationFieldDocValues, WordBreakSolrSpellCheckerTest, UnloadDistributedZkTest, JsonLoaderTest, SpatialFilterTest, TestCharFilters, QueryElevationComponentTest, ZkControllerTest, ReturnFieldsTest, TestMergePolicyConfig, TestElisionMultitermQuery, TestValueSourceCache, UpdateRequestProcessorFactoryTest, TestMaxScoreQueryParser, TestHighlightDedupGrouping, TestSurroundQueryParser, TermsComponentTest, CustomCollectionTest, HdfsBasicDistributedZk2Test, TestSort, HdfsWriteToMultipleCollectionsTest, TestBadConfig, ExpressionTest, TestFaceting, QueryEqualityTest, QueryParsingTest, PreAnalyzedFieldTest, HdfsChaosMonkeySafeLeaderTest, LeaderElectionTest, TestSearchPerf, CollectionsAPIAsyncDistributedZkTest, TestLMJelinekMercerSimilarityFactory, OpenCloseCoreStressTest, ShardRoutingTest, TestReplicationHandler, TestRandomDVFaceting, TestRealTimeGet, TestReload, DistributedTermsComponentTest, TestRangeQuery, SimpleFacetsTest, TestSolr4Spatial, StatsComponentTest, SolrCmdDistributorTest, PeerSyncTest, TestFiltering, AnalysisAfterCoreReloadTest, TestFoldingMultitermQuery, SuggesterTest, NoCacheHeaderTest, TestPseudoReturnFields, DefaultValueUpdateProcessorTest, SortByFunctionTest, DistanceFunctionTest, SolrInfoMBeanTest, CacheHeaderTest, TestQueryUtils, DirectSolrSpellCheckerTest, PrimitiveFieldTypeTest, FileBasedSpellCheckerTest, FieldAnalysisRequestHandlerTest, TestArbitraryIndexDir, RegexBoostProcessorTest, TestJmxIntegration, MBeansHandlerTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, CoreContainerCoreInitFailuresTest, TestLFUCache, UpdateParamsTest, ResponseLogComponentTest, BadComponentTest, TestBinaryField, TestConfig, OutputWriterTest, ExternalFileFieldSortTest, TestSolrCoreProperties, TestPostingsSolrHighlighter, DirectSolrConnectionTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestCodecSupport, SynonymTokenizerTest, EchoParamsTest, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TimeZoneUtilsTest, PluginInfoTest, DateMathParserTest, TestSolrXMLSerializer, PrimUtilsTest, DateFieldTest, TestUtils, ZkNodePropsTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, AbstractAnalyticsStatsTest, FieldFacetExtrasTest, FieldFacetTest, RangeFacetTest, FunctionTest, DeleteReplicaTest, DistribDocExpirationUpdateProcessorTest, OverseerStatusTest, ShardSplitTest]
   [junit4] Completed on J0 in 250.78s, 1 test, 1 error <<< FAILURES!

[...truncated 103 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1281: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:907: There were test failures: 396 suites, 1662 tests, 1 error, 37 ignored (16 assumptions)

Total time: 55 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b13 -server -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message