Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 76DC4200D17 for ; Sun, 8 Oct 2017 23:13:04 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 755531609E6; Sun, 8 Oct 2017 21:13:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id CAE811609D3 for ; Sun, 8 Oct 2017 23:13:01 +0200 (CEST) Received: (qmail 5650 invoked by uid 500); 8 Oct 2017 21:12:55 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 5639 invoked by uid 99); 8 Oct 2017 21:12:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 08 Oct 2017 21:12:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 7E70A1A0CEE for ; Sun, 8 Oct 2017 21:12:54 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.3 X-Spam-Level: X-Spam-Status: No, score=-0.3 tagged_above=-999 required=6.31 tests=[KAM_BADIPHTTP=2, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, WEIRD_PORT=0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id bHkq6bYFunIv for ; Sun, 8 Oct 2017 21:12:39 +0000 (UTC) Received: from serv1.sd-datasolutions.de (serv1.sd-datasolutions.de [88.99.242.108]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 3011B5FDDC for ; Sun, 8 Oct 2017 21:12:39 +0000 (UTC) Received: from serv1 (ip6-localhost [IPv6:::1]) by serv1.sd-datasolutions.de (Postfix) with ESMTP id 78780108003A for ; Sun, 8 Oct 2017 21:12:37 +0000 (UTC) Date: Sun, 8 Oct 2017 21:11:04 +0000 (UTC) From: Policeman Jenkins Server To: dev@lucene.apache.org Message-ID: <1891245443.49.1507497157494.JavaMail.jenkins@serv1.sd-datasolutions.de> In-Reply-To: <1879157818.29.1507463896937.JavaMail.jenkins@serv1.sd-datasolutions.de> References: <1879157818.29.1507463896937.JavaMail.jenkins@serv1.sd-datasolutions.de> Subject: [JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 4213 - Still Failing! MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_48_735541060.1507497157462" X-Jenkins-Job: Lucene-Solr-master-MacOSX X-Jenkins-Result: FAILURE archived-at: Sun, 08 Oct 2017 21:13:04 -0000 ------=_Part_48_735541060.1507497157462 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4213/ Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC 4 tests failed. FAILED: org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForImplic= itRouter Error Message: Error from server at https://127.0.0.1:58819/solr/withShardField: Async exc= eption during distributed update: Too many open files Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error= from server at https://127.0.0.1:58819/solr/withShardField: Async exceptio= n during distributed update: Too many open files =09at __randomizedtesting.SeedInfo.seed([20255BC0FBCC9D6C:7575B3525735529C]= :0) =09at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSo= lrClient.java:626) =09at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClie= nt.java:253) =09at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClie= nt.java:242) =09at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSo= lrClient.java:483) =09at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolr= Client.java:413) =09at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSo= lrClient.java:1121) =09at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnS= taleState(CloudSolrClient.java:862) =09at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrCl= ient.java:793) =09at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178= ) =09at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateReque= st.java:233) =09at org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForImplicitR= outer(CustomCollectionTest.java:141) =09at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) =09at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.= java:62) =09at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces= sorImpl.java:43) =09at java.lang.reflect.Method.invoke(Method.java:498) =09at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(Randomized= Runner.java:1737) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Random= izedRunner.java:934) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(Random= izedRunner.java:970) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(Rando= mizedRunner.java:984) =09at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$= 1.evaluate(SystemPropertiesRestoreRule.java:57) =09at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRu= leSetupTeardownChained.java:49) =09at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBef= oreAfterRule.java:45) =09at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleT= hreadAndTestName.java:48) =09at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(Test= RuleIgnoreAfterMaxFailures.java:64) =09at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFai= lure.java:47) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.= run(ThreadLeakControl.java:368) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTa= sk(ThreadLeakControl.java:817) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(Threa= dLeakControl.java:468) =09at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Ran= domizedRunner.java:943) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Random= izedRunner.java:829) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Random= izedRunner.java:879) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Random= izedRunner.java:890) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$= 1.evaluate(SystemPropertiesRestoreRule.java:57) =09at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBef= oreAfterRule.java:45) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStor= eClassName.java:41) =09at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMeth= odsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) =09at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMeth= odsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRule= AssertionsRequired.java:53) =09at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFai= lure.java:47) =09at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(Test= RuleIgnoreAfterMaxFailures.java:64) =09at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIg= noreTestSuites.java:54) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.= run(ThreadLeakControl.java:368) =09at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.solr.cloud.DistribCursorPagingTest.test Error Message: Stack Trace: java.lang.NullPointerException =09at __randomizedtesting.SeedInfo.seed([20255BC0FBCC9D6C:A871641A5530F094]= :0) =09at org.apache.solr.client.solrj.embedded.JettySolrRunner.getNodeName(Jet= tySolrRunner.java:345) =09at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(Abst= ractFullDistribZkTestBase.java:416) =09at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(Abs= tractFullDistribZkTestBase.java:334) =09at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$Shards= FixedStatement.callStatement(BaseDistributedSearchTestCase.java:991) =09at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$Shards= Statement.evaluate(BaseDistributedSearchTestCase.java:968) =09at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$= 1.evaluate(SystemPropertiesRestoreRule.java:57) =09at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRu= leSetupTeardownChained.java:49) =09at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBef= oreAfterRule.java:45) =09at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleT= hreadAndTestName.java:48) =09at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(Test= RuleIgnoreAfterMaxFailures.java:64) =09at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFai= lure.java:47) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.= run(ThreadLeakControl.java:368) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTa= sk(ThreadLeakControl.java:817) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(Threa= dLeakControl.java:468) =09at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Ran= domizedRunner.java:943) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(Random= izedRunner.java:829) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Random= izedRunner.java:879) =09at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Random= izedRunner.java:890) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$= 1.evaluate(SystemPropertiesRestoreRule.java:57) =09at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBef= oreAfterRule.java:45) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStor= eClassName.java:41) =09at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMeth= odsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) =09at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMeth= odsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRule= AssertionsRequired.java:53) =09at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFai= lure.java:47) =09at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(Test= RuleIgnoreAfterMaxFailures.java:64) =09at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIg= noreTestSuites.java:54) =09at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(St= atementAdapter.java:36) =09at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.= run(ThreadLeakControl.java:368) =09at java.lang.Thread.run(Thread.java:748) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.DistribCursorPagin= gTest Error Message: 72 threads leaked from SUITE scope at org.apache.solr.cloud.DistribCursorPa= gingTest: 1) Thread[id=3D22843, name=3DScheduledTrigger-8988-thread-3, = state=3DWAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.= Unsafe.park(Native Method) at java.util.concurrent.locks.LockSuppor= t.park(LockSupport.java:175) at java.util.concurrent.locks.Abstract= QueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:20= 39) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWork= Queue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.conc= urrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoo= lExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getT= ask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoo= lExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.conc= urrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) a= t java.lang.Thread.run(Thread.java:748) 2) Thread[id=3D22760, name=3Dqtp= 723840855-22760, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTes= t] at sun.misc.Unsafe.park(Native Method) at java.util.conc= urrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.ut= il.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(A= bstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.Bloc= kingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jet= ty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) = at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThre= adPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2= .run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java= :748) 3) Thread[id=3D22792, name=3Dqtp525107345-22792, state=3DTIMED_WAI= TING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park= (Native Method) at java.util.concurrent.locks.LockSupport.parkNanos= (LockSupport.java:215) at java.util.concurrent.locks.AbstractQueued= Synchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:207= 8) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQ= ueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.id= leJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thre= ad.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.ecl= ipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) = at java.lang.Thread.run(Thread.java:748) 4) Thread[id=3D22797, nam= e=3DConnection evictor, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPa= gingTest] at java.lang.Thread.sleep(Native Method) at org.a= pache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.ja= va:66) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=3D2= 2820, name=3Dqtp1007099615-22820, state=3DTIMED_WAITING, group=3DTGRP-Distr= ibCursorPagingTest] at sun.misc.Unsafe.park(Native Method) = at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) = at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionOb= ject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclips= e.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) a= t org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPo= ol.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.acce= ss$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.Q= ueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Threa= d.run(Thread.java:748) 6) Thread[id=3D22789, name=3Dqtp525107345-22789, = state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at sun= .misc.Unsafe.park(Native Method) at java.util.concurrent.locks.Lock= Support.parkNanos(LockSupport.java:215) at java.util.concurrent.loc= ks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSync= hronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.po= ll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.Qu= euedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclips= e.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) = at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThread= Pool.java:626) at java.lang.Thread.run(Thread.java:748) 7) Threa= d[id=3D22806, name=3DcoreContainerWorkExecutor-8970-thread-1, state=3DTIMED= _WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.= park(Native Method) at java.util.concurrent.locks.LockSupport.parkN= anos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue= $TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util= .concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:3= 62) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.= java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(Thread= PoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.= runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.Thr= eadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lan= g.Thread.run(Thread.java:748) 8) Thread[id=3D22780, name=3Dwatches-4534-= thread-1, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] = at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.= locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.conc= urrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:46= 0) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(= SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue= .poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPool= Executor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurr= ent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at j= ava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6= 24) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=3D2278= 4, name=3DConnection evictor, state=3DTIMED_WAITING, group=3DTGRP-DistribCu= rsorPagingTest] at java.lang.Thread.sleep(Native Method) at= org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvic= tor.java:66) at java.lang.Thread.run(Thread.java:748) 10) Thread[= id=3D22854, name=3DScheduledTrigger-8988-thread-4, state=3DWAITING, group= =3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Met= hod) at java.util.concurrent.locks.LockSupport.park(LockSupport.jav= a:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$Con= ditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.ut= il.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledTh= readPoolExecutor.java:1088) at java.util.concurrent.ScheduledThread= PoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) = at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor= .java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(Th= readPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecu= tor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run= (Thread.java:748) 11) Thread[id=3D22787, name=3Dqtp525107345-22787, state= =3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc= .Unsafe.park(Native Method) at java.util.concurrent.locks.LockSuppo= rt.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.Ab= stractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchroni= zer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(Bl= ockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedT= hreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jet= ty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) = at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.= java:626) at java.lang.Thread.run(Thread.java:748) 12) Thread[id= =3D22844, name=3DzkCallback-4545-thread-2, state=3DTIMED_WAITING, group=3DT= GRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method)= at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.ja= va:215) at java.util.concurrent.SynchronousQueue$TransferStack.awai= tFulfill(SynchronousQueue.java:460) at java.util.concurrent.Synchro= nousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java= .util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) a= t java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1= 073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoo= lExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Wor= ker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread= .java:748) 13) Thread[id=3D22846, name=3DzkCallback-4545-thread-4, state= =3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc= .Unsafe.park(Native Method) at java.util.concurrent.locks.LockSuppo= rt.parkNanos(LockSupport.java:215) at java.util.concurrent.Synchron= ousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at j= ava.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueu= e.java:362) at java.util.concurrent.SynchronousQueue.poll(Synchrono= usQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTas= k(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolE= xecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concur= rent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at = java.lang.Thread.run(Thread.java:748) 14) Thread[id=3D22769, name=3DConne= ction evictor, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest]= at java.lang.Thread.sleep(Native Method) at org.apache.htt= p.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) = at java.lang.Thread.run(Thread.java:748) 15) Thread[id=3D22813, nam= e=3DConnection evictor, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPa= gingTest] at java.lang.Thread.sleep(Native Method) at org.a= pache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.ja= va:66) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=3D2= 2815, name=3Dqtp1007099615-22815-acceptor-0@35cab96f-ServerConnector@721c85= d1{HTTP/1.1,[http/1.1]}{127.0.0.1:58985}, state=3DRUNNABLE, group=3DTGRP-Di= stribCursorPagingTest] at sun.nio.ch.ServerSocketChannelImpl.accept= 0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(Serve= rSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.= accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.serve= r.ServerConnector.accept(ServerConnector.java:371) at org.eclipse.j= etty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) = at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPo= ol.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.ru= n(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:74= 8) 17) Thread[id=3D22841, name=3DScheduledTrigger-8988-thread-1, state=3D= WAITING, group=3DTGRP-DistribCursorPagingTest] at java.lang.Object.= wait(Native Method) at java.lang.Object.wait(Object.java:502) = at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406) = at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:781) = at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:3= 92) at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkCl= ient.java:389) at org.apache.solr.common.cloud.ZkCmdExecutor.retryO= peration(ZkCmdExecutor.java:60) at org.apache.solr.common.cloud.Sol= rZkClient.create(SolrZkClient.java:389) at org.apache.solr.cloud.Zk= DistributedQueue.offer(ZkDistributedQueue.java:281) at org.apache.s= olr.cloud.autoscaling.TriggerEventQueue.offerEvent(TriggerEventQueue.java:5= 3) at org.apache.solr.cloud.autoscaling.ScheduledTriggers$Scheduled= Trigger.enqueue(ScheduledTriggers.java:426) at org.apache.solr.clou= d.autoscaling.ScheduledTriggers.lambda$add$3(ScheduledTriggers.java:204) = at org.apache.solr.cloud.autoscaling.ScheduledTriggers$$Lambda$296/19= 04531662.process(Unknown Source) at org.apache.solr.cloud.autoscali= ng.NodeLostTrigger.run(NodeLostTrigger.java:267) at org.apache.solr= .cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.run(ScheduledTriggers= .java:468) at java.util.concurrent.Executors$RunnableAdapter.call(E= xecutors.java:511) at java.util.concurrent.FutureTask.runAndReset(F= utureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExec= utor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) = at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureT= ask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurre= nt.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at ja= va.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:62= 4) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=3D22853= , name=3DzkCallback-4533-thread-5, state=3DTIMED_WAITING, group=3DTGRP-Dist= ribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) = at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) = at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill= (SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueu= e$TransferStack.transfer(SynchronousQueue.java:362) at java.util.co= ncurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.u= til.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) = at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto= r.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(= ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:74= 8) 19) Thread[id=3D22761, name=3Dqtp723840855-22761, state=3DTIMED_WAITIN= G, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Na= tive Method) at java.util.concurrent.locks.LockSupport.parkNanos(Lo= ckSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSyn= chronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) = at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJ= obPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.= QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclips= e.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) = at java.lang.Thread.run(Thread.java:748) 20) Thread[id=3D22809, name= =3Dwatches-4540-thread-1, state=3DTIMED_WAITING, group=3DTGRP-DistribCursor= PagingTest] at sun.misc.Unsafe.park(Native Method) at java.= util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) a= t java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Synchron= ousQueue.java:460) at java.util.concurrent.SynchronousQueue$Transfe= rStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.= SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concu= rrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at j= ava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11= 34) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo= lExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 21) = Thread[id=3D22782, name=3DsearcherExecutor-8964-thread-1, state=3DWAITING, = group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Nativ= e Method) at java.util.concurrent.locks.LockSupport.park(LockSuppor= t.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronize= r$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at ja= va.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) = at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecuto= r.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(T= hreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExec= utor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.ru= n(Thread.java:748) 22) Thread[id=3D22788, name=3Dqtp525107345-22788, stat= e=3DRUNNABLE, group=3DTGRP-DistribCursorPagingTest] at sun.nio.ch.K= QueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayW= rapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelect= orImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.Selector= Impl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorIm= pl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(S= electorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$Selec= torProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.i= o.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) = at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executePr= oduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.u= til.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceCons= ume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProd= uceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty= .util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at = org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:= 589) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=3D227= 98, name=3DConnection evictor, state=3DTIMED_WAITING, group=3DTGRP-DistribC= ursorPagingTest] at java.lang.Thread.sleep(Native Method) a= t org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvi= ctor.java:66) at java.lang.Thread.run(Thread.java:748) 24) Thread= [id=3D22803, name=3DTEST-DistribCursorPagingTest.test-seed#[20255BC0FBCC9D6= C]-EventThread, state=3DWAITING, group=3DTGRP-DistribCursorPagingTest] = at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.= locks.LockSupport.park(LockSupport.java:175) at java.util.concurren= t.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSync= hronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.tak= e(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$= EventThread.run(ClientCnxn.java:501) 25) Thread[id=3D22759, name=3Dqtp723= 840855-22759, state=3DRUNNABLE, group=3DTGRP-DistribCursorPagingTest] = at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.n= io.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.n= io.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at s= un.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun= .nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.Sel= ectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.Man= agedSelector$SelectorProducer.select(ManagedSelector.java:243) at o= rg.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelecto= r.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduc= eConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at o= rg.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(= ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.st= rategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at= org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.jav= a:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(Queu= edThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 2= 6) Thread[id=3D22818, name=3Dqtp1007099615-22818, state=3DTIMED_WAITING, gr= oup=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native = Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSup= port.java:215) at java.util.concurrent.locks.AbstractQueuedSynchron= izer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) = at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.jav= a:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPol= l(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.Queue= dThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jet= ty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at= java.lang.Thread.run(Thread.java:748) 27) Thread[id=3D22831, name=3DTEST= -DistribCursorPagingTest.test-seed#[20255BC0FBCC9D6C]-SendThread(127.0.0.1:= 58943), state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] = at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.= client.StaticHostProvider.next(StaticHostProvider.java:101) at org.= apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) = at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:106= 0) 28) Thread[id=3D22811, name=3DsearcherExecutor-8975-thread-1, state=3D= WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.p= ark(Native Method) at java.util.concurrent.locks.LockSupport.park(L= ockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSy= nchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) = at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.ja= va:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPo= olExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.ru= nWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.Threa= dPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.= Thread.run(Thread.java:748) 29) Thread[id=3D22774, name=3DTEST-DistribCur= sorPagingTest.test-seed#[20255BC0FBCC9D6C]-EventThread, state=3DWAITING, gr= oup=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native = Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.= java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$= ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java= .util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) = at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501= ) 30) Thread[id=3D22817, name=3Dqtp1007099615-22817, state=3DTIMED_WAITIN= G, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Na= tive Method) at java.util.concurrent.locks.LockSupport.parkNanos(Lo= ckSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSyn= chronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) = at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJ= obPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.= QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclips= e.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) = at java.lang.Thread.run(Thread.java:748) 31) Thread[id=3D22757, name= =3Dqtp723840855-22757-acceptor-0@7bf595b1-ServerConnector@7b054dbd{HTTP/1.1= ,[http/1.1]}{127.0.0.1:58960}, state=3DRUNNABLE, group=3DTGRP-DistribCursor= PagingTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Me= thod) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChan= nelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(Serv= erSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerCon= nector.accept(ServerConnector.java:371) at org.eclipse.jetty.server= .AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.= eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671= ) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThr= eadPool.java:589) at java.lang.Thread.run(Thread.java:748) 32) Th= read[id=3D22791, name=3Dqtp525107345-22791, state=3DTIMED_WAITING, group=3D= TGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method= ) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.j= ava:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$C= onditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at = org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)= at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queu= edThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThrea= dPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.uti= l.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.= lang.Thread.run(Thread.java:748) 33) Thread[id=3D22776, name=3DThread-395= 3, state=3DWAITING, group=3DTGRP-DistribCursorPagingTest] at java.l= ang.Object.wait(Native Method) at java.lang.Object.wait(Object.java= :502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1= 675) 34) Thread[id=3D22821, name=3Dqtp1007099615-22821, state=3DTIMED_WAI= TING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park= (Native Method) at java.util.concurrent.locks.LockSupport.parkNanos= (LockSupport.java:215) at java.util.concurrent.locks.AbstractQueued= Synchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:207= 8) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQ= ueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.id= leJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thre= ad.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.ecl= ipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) = at java.lang.Thread.run(Thread.java:748) 35) Thread[id=3D22837, nam= e=3DzkCallback-4533-thread-2, state=3DTIMED_WAITING, group=3DTGRP-DistribCu= rsorPagingTest] at sun.misc.Unsafe.park(Native Method) at j= ava.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) = at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Sync= hronousQueue.java:460) at java.util.concurrent.SynchronousQueue$Tra= nsferStack.transfer(SynchronousQueue.java:362) at java.util.concurr= ent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.c= oncurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) = at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= a:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Threa= dPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) = 36) Thread[id=3D22845, name=3DzkCallback-4545-thread-3, state=3DTIMED_WAITI= NG, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(N= ative Method) at java.util.concurrent.locks.LockSupport.parkNanos(L= ockSupport.java:215) at java.util.concurrent.SynchronousQueue$Trans= ferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concu= rrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) = at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:9= 41) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolEx= ecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWor= ker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoo= lExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thre= ad.run(Thread.java:748) 37) Thread[id=3D22786, name=3Dqtp525107345-22786-= acceptor-0@6483d2d7-ServerConnector@7bf05096{HTTP/1.1,[http/1.1]}{127.0.0.1= :58967}, state=3DRUNNABLE, group=3DTGRP-DistribCursorPagingTest] at= sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.n= io.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) = at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.ja= va:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerCo= nnector.java:371) at org.eclipse.jetty.server.AbstractConnector$Acc= eptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thr= ead.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclip= se.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) = at java.lang.Thread.run(Thread.java:748) 38) Thread[id=3D22793, name= =3Dorg.eclipse.jetty.server.session.HashSessionManager@5f9f6763Timer, state= =3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc= .Unsafe.park(Native Method) at java.util.concurrent.locks.LockSuppo= rt.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.Ab= stractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchroni= zer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$= DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at jav= a.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Schedul= edThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolEx= ecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurren= t.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at jav= a.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624= ) at java.lang.Thread.run(Thread.java:748) 39) Thread[id=3D22808,= name=3DScheduler-1690292308, state=3DTIMED_WAITING, group=3DTGRP-DistribCu= rsorPagingTest] at sun.misc.Unsafe.park(Native Method) at j= ava.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) = at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject= .awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.conc= urrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoo= lExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExe= cutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) a= t java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1= 074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoo= lExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Wor= ker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread= .java:748) 40) Thread[id=3D22822, name=3Dorg.eclipse.jetty.server.session= .HashSessionManager@30ff273bTimer, state=3DTIMED_WAITING, group=3DTGRP-Dist= ribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) = at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) = at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util= .concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThre= adPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPo= olExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) = at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.j= ava:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(Thre= adPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecuto= r$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(T= hread.java:748) 41) Thread[id=3D22775, name=3DzkCallback-4533-thread-1, s= tate=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.= misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockS= upport.parkNanos(LockSupport.java:215) at java.util.concurrent.Sync= hronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) = at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Synchronous= Queue.java:362) at java.util.concurrent.SynchronousQueue.poll(Synch= ronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.ge= tTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadP= oolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.co= ncurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) = at java.lang.Thread.run(Thread.java:748) 42) Thread[id=3D22756, name=3Dq= tp723840855-22756, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingT= est] at sun.misc.Unsafe.park(Native Method) at java.util.co= ncurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.= util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos= (AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.Bl= ockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.j= etty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) = at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedTh= readPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool= $2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.ja= va:748) 43) Thread[id=3D22819, name=3Dqtp1007099615-22819, state=3DTIMED_= WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.p= ark(Native Method) at java.util.concurrent.locks.LockSupport.parkNa= nos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQue= uedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:= 2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArr= ayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool= .idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.t= hread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.= eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)= at java.lang.Thread.run(Thread.java:748) 44) Thread[id=3D22777, = name=3DcoreContainerWorkExecutor-8959-thread-1, state=3DTIMED_WAITING, grou= p=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Me= thod) at java.util.concurrent.locks.LockSupport.parkNanos(LockSuppo= rt.java:215) at java.util.concurrent.SynchronousQueue$TransferStack= .awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.Sy= nchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at= java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) = at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.j= ava:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(Thre= adPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecuto= r$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(T= hread.java:748) 45) Thread[id=3D22816, name=3Dqtp1007099615-22816, state= =3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc= .Unsafe.park(Native Method) at java.util.concurrent.locks.LockSuppo= rt.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.Ab= stractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchroni= zer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(Bl= ockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedT= hreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jet= ty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) = at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.= java:626) at java.lang.Thread.run(Thread.java:748) 46) Thread[id= =3D22785, name=3Dqtp525107345-22785, state=3DTIMED_WAITING, group=3DTGRP-Di= stribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) = at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215= ) at java.util.concurrent.locks.AbstractQueuedSynchronizer$Conditio= nObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.ecl= ipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) = at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThrea= dPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.a= ccess$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.threa= d.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Th= read.run(Thread.java:748) 47) Thread[id=3D22763, name=3Dqtp723840855-2276= 3, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at = sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.L= ockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.= locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedS= ynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue= .poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread= .QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.ecl= ipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48= ) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThr= eadPool.java:626) at java.lang.Thread.run(Thread.java:748) 48) Th= read[id=3D22814, name=3Dqtp1007099615-22814, state=3DRUNNABLE, group=3DTGRP= -DistribCursorPagingTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(= Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWra= pper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSele= ctorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(Selec= torImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.jav= a:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) = at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(Managed= Selector.java:243) at org.eclipse.jetty.io.ManagedSelector$Selector= Producer.produce(ManagedSelector.java:191) at org.eclipse.jetty.uti= l.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduc= eConsume.java:249) at org.eclipse.jetty.util.thread.strategy.Execut= eProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at o= rg.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(Execute= ProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.ru= n(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.Queued= ThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.u= til.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at jav= a.lang.Thread.run(Thread.java:748) 49) Thread[id=3D22840, name=3DOverseer= AutoScalingTriggerThread-98795252130906121-127.0.0.1:58960_jqsjx-n_00000000= 01, state=3DWAITING, group=3DOverseer autoscaling triggers] at sun.= misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockS= upport.park(LockSupport.java:175) at java.util.concurrent.locks.Abs= tractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.ja= va:2039) at org.apache.solr.cloud.autoscaling.OverseerTriggerThread= .run(OverseerTriggerThread.java:168) at java.lang.Thread.run(Thread= .java:748) 50) Thread[id=3D22758, name=3Dqtp723840855-22758, state=3DTIME= D_WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe= .park(Native Method) at java.util.concurrent.locks.LockSupport.park= Nanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQ= ueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.jav= a:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingA= rrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPo= ol.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util= .thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at or= g.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:62= 6) at java.lang.Thread.run(Thread.java:748) 51) Thread[id=3D22848= , name=3DzkCallback-4539-thread-2, state=3DTIMED_WAITING, group=3DTGRP-Dist= ribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) = at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) = at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill= (SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueu= e$TransferStack.transfer(SynchronousQueue.java:362) at java.util.co= ncurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.u= til.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) = at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto= r.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(= ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:74= 8) 52) Thread[id=3D22832, name=3DTEST-DistribCursorPagingTest.test-seed#[= 20255BC0FBCC9D6C]-EventThread, state=3DWAITING, group=3DTGRP-DistribCursorP= agingTest] at sun.misc.Unsafe.park(Native Method) at java.u= til.concurrent.locks.LockSupport.park(LockSupport.java:175) at java= .util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Abs= tractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBl= ockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookee= per.ClientCnxn$EventThread.run(ClientCnxn.java:501) 53) Thread[id=3D22827= , name=3DConnection evictor, state=3DTIMED_WAITING, group=3DTGRP-DistribCur= sorPagingTest] at java.lang.Thread.sleep(Native Method) at = org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvict= or.java:66) at java.lang.Thread.run(Thread.java:748) 54) Thread[i= d=3D22764, name=3Dorg.eclipse.jetty.server.session.HashSessionManager@3ef81= 6aeTimer, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] = at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.= locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.conc= urrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Abstract= QueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThre= adPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)= at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQue= ue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurre= nt.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java= .util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)= at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolEx= ecutor.java:624) at java.lang.Thread.run(Thread.java:748) 55) Thr= ead[id=3D22762, name=3Dqtp723840855-22762, state=3DTIMED_WAITING, group=3DT= GRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method)= at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.ja= va:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$Co= nditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at o= rg.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) = at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queue= dThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThread= Pool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util= .thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.l= ang.Thread.run(Thread.java:748) 56) Thread[id=3D22779, name=3DScheduler-1= 667037032, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] = at sun.misc.Unsafe.park(Native Method) at java.util.concurrent= .locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.con= current.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Abstrac= tQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThr= eadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093= ) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQu= eue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurr= ent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at jav= a.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134= ) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:624) at java.lang.Thread.run(Thread.java:748) 57) Th= read[id=3D22842, name=3DScheduledTrigger-8988-thread-2, state=3DWAITING, gr= oup=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native = Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.= java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$= ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java= .util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Schedule= dThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThr= eadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)= at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecu= tor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker= (ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolEx= ecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.= run(Thread.java:748) 58) Thread[id=3D22852, name=3DzkCallback-4539-thread= -5, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at= sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.= LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent= .SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) = at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Synchr= onousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(= SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecut= or.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.Th= readPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.ut= il.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) = at java.lang.Thread.run(Thread.java:748) 59) Thread[id=3D22833, nam= e=3DzkCallback-4545-thread-1, state=3DTIMED_WAITING, group=3DTGRP-DistribCu= rsorPagingTest] at sun.misc.Unsafe.park(Native Method) at j= ava.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) = at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Sync= hronousQueue.java:460) at java.util.concurrent.SynchronousQueue$Tra= nsferStack.transfer(SynchronousQueue.java:362) at java.util.concurr= ent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.c= oncurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) = at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav= a:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Threa= dPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) = 60) Thread[id=3D22805, name=3DThread-3962, state=3DWAITING, group=3DTGRP-Di= stribCursorPagingTest] at java.lang.Object.wait(Native Method) = at java.lang.Object.wait(Object.java:502) at org.apache.solr.co= re.CloserThread.run(CoreContainer.java:1675) 61) Thread[id=3D22826, name= =3DConnection evictor, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPag= ingTest] at java.lang.Thread.sleep(Native Method) at org.ap= ache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.jav= a:66) at java.lang.Thread.run(Thread.java:748) 62) Thread[id=3D22= 790, name=3Dqtp525107345-22790, state=3DTIMED_WAITING, group=3DTGRP-Distrib= CursorPagingTest] at sun.misc.Unsafe.park(Native Method) at= java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) = at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObje= ct.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.= jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at = org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool= .java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access= $800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.Que= uedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.= run(Thread.java:748) 63) Thread[id=3D22838, name=3DOverseerStateUpdate-98= 795252130906121-127.0.0.1:58960_jqsjx-n_0000000001, state=3DTIMED_WAITING, = group=3DOverseer state updater.] at sun.misc.Unsafe.park(Native Met= hod) at java.util.concurrent.locks.LockSupport.parkNanos(LockSuppor= t.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronize= r$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) = at org.apache.solr.cloud.ZkDistributedQueue.peek(ZkDistributedQueue.java:18= 3) at org.apache.solr.cloud.ZkDistributedQueue.peek(ZkDistributedQu= eue.java:156) at org.apache.solr.cloud.Overseer$ClusterStateUpdater= .run(Overseer.java:186) at java.lang.Thread.run(Thread.java:748) = 64) Thread[id=3D22839, name=3DOverseerCollectionConfigSetProcessor-98795252= 130906121-127.0.0.1:58960_jqsjx-n_0000000001, state=3DTIMED_WAITING, group= =3DOverseer collection creation process.] at java.lang.Thread.sleep= (Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retry= Delay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmd= Executor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.c= ommon.cloud.SolrZkClient.getData(SolrZkClient.java:354) at org.apac= he.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:38= 7) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskP= rocessor.java:193) at java.lang.Thread.run(Thread.java:748) 65) T= hread[id=3D22804, name=3DzkCallback-4539-thread-1, state=3DTIMED_WAITING, g= roup=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native= Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSu= pport.java:215) at java.util.concurrent.SynchronousQueue$TransferSt= ack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent= .SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) = at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) = at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecuto= r.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(T= hreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExec= utor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.ru= n(Thread.java:748) 66) Thread[id=3D22851, name=3DzkCallback-4539-thread-4= , state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at s= un.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.Lo= ckSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.S= ynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) = at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Synchron= ousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(Sy= nchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor= .getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.Thre= adPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util= .concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) = at java.lang.Thread.run(Thread.java:748) 67) Thread[id=3D22773, name= =3DTEST-DistribCursorPagingTest.test-seed#[20255BC0FBCC9D6C]-SendThread(127= .0.0.1:58943), state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest]= at java.lang.Thread.sleep(Native Method) at org.apache.zoo= keeper.client.StaticHostProvider.next(StaticHostProvider.java:101) = at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:= 997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.j= ava:1060) 68) Thread[id=3D22849, name=3DzkCallback-4533-thread-4, state= =3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc= .Unsafe.park(Native Method) at java.util.concurrent.locks.LockSuppo= rt.parkNanos(LockSupport.java:215) at java.util.concurrent.Synchron= ousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at j= ava.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueu= e.java:362) at java.util.concurrent.SynchronousQueue.poll(Synchrono= usQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTas= k(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolE= xecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concur= rent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at = java.lang.Thread.run(Thread.java:748) 69) Thread[id=3D22850, name=3DzkCal= lback-4539-thread-3, state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagin= gTest] at sun.misc.Unsafe.park(Native Method) at java.util.= concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at jav= a.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQu= eue.java:460) at java.util.concurrent.SynchronousQueue$TransferStac= k.transfer(SynchronousQueue.java:362) at java.util.concurrent.Synch= ronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent= .ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.u= til.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) = at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExec= utor.java:624) at java.lang.Thread.run(Thread.java:748) 70) Threa= d[id=3D22768, name=3DConnection evictor, state=3DTIMED_WAITING, group=3DTGR= P-DistribCursorPagingTest] at java.lang.Thread.sleep(Native Method)= at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleCon= nectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) = 71) Thread[id=3D22847, name=3DzkCallback-4533-thread-3, state=3DTIMED_WAITI= NG, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(N= ative Method) at java.util.concurrent.locks.LockSupport.parkNanos(L= ockSupport.java:215) at java.util.concurrent.SynchronousQueue$Trans= ferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concu= rrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) = at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:9= 41) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolEx= ecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWor= ker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoo= lExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thre= ad.run(Thread.java:748) 72) Thread[id=3D22802, name=3DTEST-DistribCursorP= agingTest.test-seed#[20255BC0FBCC9D6C]-SendThread(127.0.0.1:58943), state= =3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at java.lan= g.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$Se= ndThread.run(ClientCnxn.java:1051) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 72 threads leaked from = SUITE scope at org.apache.solr.cloud.DistribCursorPagingTest:=20 1) Thread[id=3D22843, name=3DScheduledTrigger-8988-thread-3, state=3DWAI= TING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175= ) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueu= e.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueu= e.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecut= or.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=3D22760, name=3Dqtp723840855-22760, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queue= dThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(Queued= ThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:626) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=3D22792, name=3Dqtp525107345-22792, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queue= dThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(Queued= ThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:626) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=3D22797, name=3DConnection evictor, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConn= ectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=3D22820, name=3Dqtp1007099615-22820, state=3DTIMED_WAITING,= group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queue= dThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(Queued= ThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:626) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=3D22789, name=3Dqtp525107345-22789, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queue= dThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(Queued= ThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:626) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=3D22806, name=3DcoreContainerWorkExecutor-8970-thread-1, st= ate=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill= (SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Syn= chronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java= :941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecut= or.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=3D22780, name=3Dwatches-4534-thread-1, state=3DTIMED_WAITIN= G, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill= (SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Syn= chronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java= :941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecut= or.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=3D22784, name=3DConnection evictor, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConn= ectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=3D22854, name=3DScheduledTrigger-8988-thread-4, state=3DWAI= TING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175= ) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueu= e.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueu= e.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecut= or.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=3D22787, name=3Dqtp525107345-22787, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queue= dThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(Queued= ThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:626) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=3D22844, name=3DzkCallback-4545-thread-2, state=3DTIMED_WAI= TING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill= (SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Syn= chronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java= :941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecut= or.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=3D22846, name=3DzkCallback-4545-thread-4, state=3DTIMED_WAI= TING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill= (SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Syn= chronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java= :941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecut= or.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 14) Thread[id=3D22769, name=3DConnection evictor, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConn= ectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=3D22813, name=3DConnection evictor, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConn= ectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=3D22815, name=3Dqtp1007099615-22815-acceptor-0@35cab96f-Ser= verConnector@721c85d1{HTTP/1.1,[http/1.1]}{127.0.0.1:58985}, state=3DRUNNAB= LE, group=3DTGRP-DistribCursorPagingTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImp= l.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImp= l.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.= java:371) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(Abstract= Connector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThre= adPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:589) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=3D22841, name=3DScheduledTrigger-8988-thread-1, state=3DWAI= TING, group=3DTGRP-DistribCursorPagingTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:14= 06) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:781) at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient= .java:392) at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient= .java:389) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdE= xecutor.java:60) at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.ja= va:389) at org.apache.solr.cloud.ZkDistributedQueue.offer(ZkDistributedQueu= e.java:281) at org.apache.solr.cloud.autoscaling.TriggerEventQueue.offerEvent(T= riggerEventQueue.java:53) at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTri= gger.enqueue(ScheduledTriggers.java:426) at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$3= (ScheduledTriggers.java:204) at org.apache.solr.cloud.autoscaling.ScheduledTriggers$$Lambda$296/= 1904531662.process(Unknown Source) at org.apache.solr.cloud.autoscaling.NodeLostTrigger.run(NodeLostTr= igger.java:267) at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTri= gger.run(ScheduledTriggers.java:468) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.ja= va:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuture= Task.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuture= Task.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=3D22853, name=3DzkCallback-4533-thread-5, state=3DTIMED_WAI= TING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill= (SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Syn= chronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java= :941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecut= or.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 19) Thread[id=3D22761, name=3Dqtp723840855-22761, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queue= dThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(Queued= ThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:626) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=3D22809, name=3Dwatches-4540-thread-1, state=3DTIMED_WAITIN= G, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill= (SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Syn= chronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java= :941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecut= or.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 21) Thread[id=3D22782, name=3DsearcherExecutor-8964-thread-1, state=3DWAI= TING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175= ) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueu= e.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecut= or.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 22) Thread[id=3D22788, name=3Dqtp525107345-22788, state=3DRUNNABLE, group= =3DTGRP-DistribCursorPagingTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:1= 17) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(Man= agedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(Ma= nagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.exe= cuteProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.pro= duceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run= (ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThre= adPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:589) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=3D22798, name=3DConnection evictor, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConn= ectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 24) Thread[id=3D22803, name=3DTEST-DistribCursorPagingTest.test-seed#[202= 55BC0FBCC9D6C]-EventThread, state=3DWAITING, group=3DTGRP-DistribCursorPagi= ngTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175= ) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueu= e.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) 25) Thread[id=3D22759, name=3Dqtp723840855-22759, state=3DRUNNABLE, group= =3DTGRP-DistribCursorPagingTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:1= 17) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(Man= agedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(Ma= nagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.exe= cuteProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.pro= duceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run= (ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThre= adPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:589) at java.lang.Thread.run(Thread.java:748) 26) Thread[id=3D22818, name=3Dqtp1007099615-22818, state=3DTIMED_WAITING,= group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queue= dThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(Queued= ThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:626) at java.lang.Thread.run(Thread.java:748) 27) Thread[id=3D22831, name=3DTEST-DistribCursorPagingTest.test-seed#[202= 55BC0FBCC9D6C]-SendThread(127.0.0.1:58943), state=3DTIMED_WAITING, group=3D= TGRP-DistribCursorPagingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostPr= ovider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCn= xn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 060) 28) Thread[id=3D22811, name=3DsearcherExecutor-8975-thread-1, state=3DWAI= TING, group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175= ) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueu= e.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecut= or.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor.java:624) at java.lang.Thread.run(Thread.java:748) 29) Thread[id=3D22774, name=3DTEST-DistribCursorPagingTest.test-seed#[202= 55BC0FBCC9D6C]-EventThread, state=3DWAITING, group=3DTGRP-DistribCursorPagi= ngTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175= ) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueu= e.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:= 501) 30) Thread[id=3D22817, name=3Dqtp1007099615-22817, state=3DTIMED_WAITING,= group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queue= dThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(Queued= ThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:626) at java.lang.Thread.run(Thread.java:748) 31) Thread[id=3D22757, name=3Dqtp723840855-22757-acceptor-0@7bf595b1-Serv= erConnector@7b054dbd{HTTP/1.1,[http/1.1]}{127.0.0.1:58960}, state=3DRUNNABL= E, group=3DTGRP-DistribCursorPagingTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImp= l.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImp= l.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.= java:371) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(Abstract= Connector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThre= adPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:589) at java.lang.Thread.run(Thread.java:748) 32) Thread[id=3D22791, name=3Dqtp525107345-22791, state=3DTIMED_WAITING, = group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionO= bject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueu= e.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(Queue= dThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(Queued= ThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThrea= dPool.java:626) at java.lang.Thread.run(Thread.java:748) 33) Thread[id=3D22776, name=3DThread-3953, state=3DWAITING, group=3DTGRP-= DistribCursorPagingTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1675) 34) Thread[id=3D22821, name=3Dqtp1007099615-22821, state=3DTIMED_WAITING,= group=3DTGRP-DistribCursorPagingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.jav= a:215) at jav [...truncated too long message...] ursorPagingTest] [junit4] > at java.lang.Thread.sleep(Native Method) [junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(= ClientCnxn.java:1051) [junit4] > 28) Thread[id=3D22759, name=3Dqtp723840855-22759, state= =3DRUNNABLE, group=3DTGRP-DistribCursorPagingTest] [junit4] > at sun.nio.ch.KQueueArrayWrapper.interrupt(Native = Method) [junit4] > at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueA= rrayWrapper.java:203) [junit4] > at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSele= ctorImpl.java:257) [junit4] > at java.nio.channels.spi.AbstractSelector$1.interr= upt(AbstractSelector.java:213) [junit4] > at java.nio.channels.spi.AbstractSelector.begin(Ab= stractSelector.java:219) [junit4] > at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSe= lectorImpl.java:116) [junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(Selecto= rImpl.java:86) [junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.jav= a:97) [junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.jav= a:101) [junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorPr= oducer.select(ManagedSelector.java:243) [junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorPr= oducer.produce(ManagedSelector.java:191) [junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteP= roduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) [junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteP= roduceConsume.produceConsume(ExecuteProduceConsume.java:148) [junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteP= roduceConsume.run(ExecuteProduceConsume.java:136) [junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.= runJob(QueuedThreadPool.java:671) [junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$= 2.run(QueuedThreadPool.java:589) [junit4] > at java.lang.Thread.run(Thread.java:748) [junit4] > 29) Thread[id=3D22849, name=3DzkCallback-4533-thread-4, = state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNano= s(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferS= tack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferS= tack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(Sync= hronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask= (ThreadPoolExecutor.java:1073) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWork= er(ThreadPoolExecutor.java:1134) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.= run(ThreadPoolExecutor.java:624) [junit4] > at java.lang.Thread.run(Thread.java:748) [junit4] > 30) Thread[id=3D22831, name=3DTEST-DistribCursorPagingTe= st.test-seed#[20255BC0FBCC9D6C]-SendThread(127.0.0.1:58943), state=3DTIMED_= WAITING, group=3DTGRP-DistribCursorPagingTest] [junit4] > at java.lang.Thread.sleep(Native Method) [junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(= ClientCnxn.java:1051) [junit4] > 31) Thread[id=3D22811, name=3DsearcherExecutor-8975-thre= ad-1, state=3DWAITING, group=3DTGRP-DistribCursorPagingTest] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.park(Loc= kSupport.java:175) [junit4] > at java.util.concurrent.locks.AbstractQueuedSynchr= onizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) [junit4] > at java.util.concurrent.LinkedBlockingQueue.take(L= inkedBlockingQueue.java:442) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask= (ThreadPoolExecutor.java:1074) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWork= er(ThreadPoolExecutor.java:1134) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.= run(ThreadPoolExecutor.java:624) [junit4] > at java.lang.Thread.run(Thread.java:748) [junit4] > 32) Thread[id=3D22850, name=3DzkCallback-4539-thread-3, = state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNano= s(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferS= tack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferS= tack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(Sync= hronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask= (ThreadPoolExecutor.java:1073) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWork= er(ThreadPoolExecutor.java:1134) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.= run(ThreadPoolExecutor.java:624) [junit4] > at java.lang.Thread.run(Thread.java:748) [junit4] > 33) Thread[id=3D22847, name=3DzkCallback-4533-thread-3, = state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNano= s(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferS= tack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferS= tack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(Sync= hronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask= (ThreadPoolExecutor.java:1073) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWork= er(ThreadPoolExecutor.java:1134) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.= run(ThreadPoolExecutor.java:624) [junit4] > at java.lang.Thread.run(Thread.java:748) [junit4] > 34) Thread[id=3D22776, name=3DThread-3953, state=3DWAITI= NG, group=3DTGRP-DistribCursorPagingTest] [junit4] > at java.lang.Object.wait(Native Method) [junit4] > at java.lang.Object.wait(Object.java:502) [junit4] > at org.apache.solr.core.CloserThread.run(CoreConta= iner.java:1675) [junit4] > 35) Thread[id=3D22802, name=3DTEST-DistribCursorPagingTe= st.test-seed#[20255BC0FBCC9D6C]-SendThread(127.0.0.1:58943), state=3DTIMED_= WAITING, group=3DTGRP-DistribCursorPagingTest] [junit4] > at java.lang.Thread.sleep(Native Method) [junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(= ClientCnxn.java:1051) [junit4] > 36) Thread[id=3D22837, name=3DzkCallback-4533-thread-2, = state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNano= s(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferS= tack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferS= tack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(Sync= hronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask= (ThreadPoolExecutor.java:1073) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWork= er(ThreadPoolExecutor.java:1134) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.= run(ThreadPoolExecutor.java:624) [junit4] > at java.lang.Thread.run(Thread.java:748) [junit4] > 37) Thread[id=3D22845, name=3DzkCallback-4545-thread-3, = state=3DTIMED_WAITING, group=3DTGRP-DistribCursorPagingTest] [junit4] > at sun.misc.Unsafe.park(Native Method) [junit4] > at java.util.concurrent.locks.LockSupport.parkNano= s(LockSupport.java:215) [junit4] > at java.util.concurrent.SynchronousQueue$TransferS= tack.awaitFulfill(SynchronousQueue.java:460) [junit4] > at java.util.concurrent.SynchronousQueue$TransferS= tack.transfer(SynchronousQueue.java:362) [junit4] > at java.util.concurrent.SynchronousQueue.poll(Sync= hronousQueue.java:941) [junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask= (ThreadPoolExecutor.java:1073) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWork= er(ThreadPoolExecutor.java:1134) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.= run(ThreadPoolExecutor.java:624) [junit4] > at java.lang.Thread.run(Thread.java:748) [junit4] > =09at __randomizedtesting.SeedInfo.seed([20255BC0FBCC9D6C]= :0) [junit4] Completed [516/745 (2!)] on J0 in 33.56s, 1 test, 3 errors <<< = FAILURES! [...truncated 47582 lines...] BUILD FAILED /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:826: The follo= wing error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:706: The follo= wing error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:693: Source ch= eckout is dirty (unversioned/missing files) after running tests!!! Offendin= g files: * lucene/licenses/morfologik-ukrainian-search-3.7.5.jar.sha1 Total time: 72 minutes 53 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts [WARNINGS] Skipping publisher since build result is FAILURE Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ------=_Part_48_735541060.1507497157462 Content-Type: text/plain; charset=us-ascii --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org ------=_Part_48_735541060.1507497157462--