lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Jenkins Server <jenk...@builds.apache.org>
Subject [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 85 - Still Failing
Date Sun, 04 Nov 2012 22:19:27 GMT
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/85/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping:     1) Thread[id=523, name=qtp2121902167-523 Acceptor2 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    2) Thread[id=200, name=qtp1313447888-200 Acceptor1 SelectChannelConnector@0.0.0.0:58411, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    3) Thread[id=362, name=qtp161733249-362 Acceptor2 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    4) Thread[id=360, name=qtp161733249-360 Acceptor0 SelectChannelConnector@0.0.0.0:58427, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    5) Thread[id=522, name=qtp2121902167-522 Acceptor1 SelectChannelConnector@0.0.0.0:49081, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    6) Thread[id=524, name=qtp2121902167-524 Acceptor3 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    7) Thread[id=201, name=qtp1313447888-201 Acceptor2 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    8) Thread[id=334, name=qtp1877622224-334 Acceptor2 SelectChannelConnector@0.0.0.0:58424, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    9) Thread[id=348, name=qtp1100289413-348 Acceptor2 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   10) Thread[id=493, name=qtp1376091878-493 Acceptor0 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   11) Thread[id=202, name=qtp1313447888-202 Acceptor3 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   12) Thread[id=199, name=qtp1313447888-199 Acceptor0 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   13) Thread[id=495, name=qtp1376091878-495 Acceptor2 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   14) Thread[id=496, name=qtp1376091878-496 Acceptor3 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   15) Thread[id=215, name=qtp1734092677-215 Acceptor3 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   16) Thread[id=214, name=qtp1734092677-214 Acceptor2 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   17) Thread[id=361, name=qtp161733249-361 Acceptor1 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   18) Thread[id=346, name=qtp1100289413-346 Acceptor0 SelectChannelConnector@0.0.0.0:58425, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   19) Thread[id=347, name=qtp1100289413-347 Acceptor1 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   20) Thread[id=332, name=qtp1877622224-332 Acceptor0 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   21) Thread[id=509, name=qtp1588150803-509 Acceptor2 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   22) Thread[id=335, name=qtp1877622224-335 Acceptor3 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   23) Thread[id=349, name=qtp1100289413-349 Acceptor3 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   24) Thread[id=508, name=qtp1588150803-508 Acceptor1 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   25) Thread[id=212, name=qtp1734092677-212 Acceptor0 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   26) Thread[id=363, name=qtp161733249-363 Acceptor3 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   27) Thread[id=535, name=qtp669515876-535 Acceptor1 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   28) Thread[id=507, name=qtp1588150803-507 Acceptor0 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   29) Thread[id=534, name=qtp669515876-534 Acceptor0 SelectChannelConnector@0.0.0.0:39455, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   30) Thread[id=510, name=qtp1588150803-510 Acceptor3 SelectChannelConnector@0.0.0.0:12040, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   31) Thread[id=537, name=qtp669515876-537 Acceptor3 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   32) Thread[id=213, name=qtp1734092677-213 Acceptor1 SelectChannelConnector@0.0.0.0:58413, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   33) Thread[id=333, name=qtp1877622224-333 Acceptor1 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   34) Thread[id=521, name=qtp2121902167-521 Acceptor0 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   35) Thread[id=536, name=qtp669515876-536 Acceptor2 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   36) Thread[id=494, name=qtp1376091878-494 Acceptor1 SelectChannelConnector@0.0.0.0:59385, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping: 
   1) Thread[id=523, name=qtp2121902167-523 Acceptor2 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   2) Thread[id=200, name=qtp1313447888-200 Acceptor1 SelectChannelConnector@0.0.0.0:58411, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   3) Thread[id=362, name=qtp161733249-362 Acceptor2 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   4) Thread[id=360, name=qtp161733249-360 Acceptor0 SelectChannelConnector@0.0.0.0:58427, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   5) Thread[id=522, name=qtp2121902167-522 Acceptor1 SelectChannelConnector@0.0.0.0:49081, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   6) Thread[id=524, name=qtp2121902167-524 Acceptor3 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   7) Thread[id=201, name=qtp1313447888-201 Acceptor2 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   8) Thread[id=334, name=qtp1877622224-334 Acceptor2 SelectChannelConnector@0.0.0.0:58424, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   9) Thread[id=348, name=qtp1100289413-348 Acceptor2 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  10) Thread[id=493, name=qtp1376091878-493 Acceptor0 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  11) Thread[id=202, name=qtp1313447888-202 Acceptor3 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  12) Thread[id=199, name=qtp1313447888-199 Acceptor0 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  13) Thread[id=495, name=qtp1376091878-495 Acceptor2 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  14) Thread[id=496, name=qtp1376091878-496 Acceptor3 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  15) Thread[id=215, name=qtp1734092677-215 Acceptor3 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  16) Thread[id=214, name=qtp1734092677-214 Acceptor2 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  17) Thread[id=361, name=qtp161733249-361 Acceptor1 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  18) Thread[id=346, name=qtp1100289413-346 Acceptor0 SelectChannelConnector@0.0.0.0:58425, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  19) Thread[id=347, name=qtp1100289413-347 Acceptor1 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  20) Thread[id=332, name=qtp1877622224-332 Acceptor0 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  21) Thread[id=509, name=qtp1588150803-509 Acceptor2 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  22) Thread[id=335, name=qtp1877622224-335 Acceptor3 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  23) Thread[id=349, name=qtp1100289413-349 Acceptor3 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  24) Thread[id=508, name=qtp1588150803-508 Acceptor1 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  25) Thread[id=212, name=qtp1734092677-212 Acceptor0 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  26) Thread[id=363, name=qtp161733249-363 Acceptor3 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  27) Thread[id=535, name=qtp669515876-535 Acceptor1 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  28) Thread[id=507, name=qtp1588150803-507 Acceptor0 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  29) Thread[id=534, name=qtp669515876-534 Acceptor0 SelectChannelConnector@0.0.0.0:39455, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  30) Thread[id=510, name=qtp1588150803-510 Acceptor3 SelectChannelConnector@0.0.0.0:12040, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  31) Thread[id=537, name=qtp669515876-537 Acceptor3 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  32) Thread[id=213, name=qtp1734092677-213 Acceptor1 SelectChannelConnector@0.0.0.0:58413, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  33) Thread[id=333, name=qtp1877622224-333 Acceptor1 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  34) Thread[id=521, name=qtp2121902167-521 Acceptor0 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  35) Thread[id=536, name=qtp669515876-536 Acceptor2 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  36) Thread[id=494, name=qtp1376091878-494 Acceptor1 SelectChannelConnector@0.0.0.0:59385, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
	at __randomizedtesting.SeedInfo.seed([640759F856731E77]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=523, name=qtp2121902167-523 Acceptor2 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    2) Thread[id=200, name=qtp1313447888-200 Acceptor1 SelectChannelConnector@0.0.0.0:58411, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    3) Thread[id=362, name=qtp161733249-362 Acceptor2 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    4) Thread[id=360, name=qtp161733249-360 Acceptor0 SelectChannelConnector@0.0.0.0:58427, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    5) Thread[id=522, name=qtp2121902167-522 Acceptor1 SelectChannelConnector@0.0.0.0:49081, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    6) Thread[id=524, name=qtp2121902167-524 Acceptor3 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    7) Thread[id=201, name=qtp1313447888-201 Acceptor2 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    8) Thread[id=334, name=qtp1877622224-334 Acceptor2 SelectChannelConnector@0.0.0.0:58424, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    9) Thread[id=348, name=qtp1100289413-348 Acceptor2 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   10) Thread[id=493, name=qtp1376091878-493 Acceptor0 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   11) Thread[id=202, name=qtp1313447888-202 Acceptor3 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   12) Thread[id=199, name=qtp1313447888-199 Acceptor0 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   13) Thread[id=495, name=qtp1376091878-495 Acceptor2 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   14) Thread[id=496, name=qtp1376091878-496 Acceptor3 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   15) Thread[id=214, name=qtp1734092677-214 Acceptor2 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   16) Thread[id=215, name=qtp1734092677-215 Acceptor3 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   17) Thread[id=361, name=qtp161733249-361 Acceptor1 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   18) Thread[id=346, name=qtp1100289413-346 Acceptor0 SelectChannelConnector@0.0.0.0:58425, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   19) Thread[id=347, name=qtp1100289413-347 Acceptor1 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   20) Thread[id=332, name=qtp1877622224-332 Acceptor0 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   21) Thread[id=335, name=qtp1877622224-335 Acceptor3 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   22) Thread[id=509, name=qtp1588150803-509 Acceptor2 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   23) Thread[id=349, name=qtp1100289413-349 Acceptor3 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   24) Thread[id=212, name=qtp1734092677-212 Acceptor0 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   25) Thread[id=508, name=qtp1588150803-508 Acceptor1 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   26) Thread[id=363, name=qtp161733249-363 Acceptor3 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   27) Thread[id=535, name=qtp669515876-535 Acceptor1 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   28) Thread[id=507, name=qtp1588150803-507 Acceptor0 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   29) Thread[id=534, name=qtp669515876-534 Acceptor0 SelectChannelConnector@0.0.0.0:39455, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   30) Thread[id=510, name=qtp1588150803-510 Acceptor3 SelectChannelConnector@0.0.0.0:12040, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   31) Thread[id=537, name=qtp669515876-537 Acceptor3 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   32) Thread[id=213, name=qtp1734092677-213 Acceptor1 SelectChannelConnector@0.0.0.0:58413, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   33) Thread[id=333, name=qtp1877622224-333 Acceptor1 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   34) Thread[id=521, name=qtp2121902167-521 Acceptor0 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   35) Thread[id=536, name=qtp669515876-536 Acceptor2 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)   36) Thread[id=494, name=qtp1376091878-494 Acceptor1 SelectChannelConnector@0.0.0.0:59385, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=523, name=qtp2121902167-523 Acceptor2 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   2) Thread[id=200, name=qtp1313447888-200 Acceptor1 SelectChannelConnector@0.0.0.0:58411, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   3) Thread[id=362, name=qtp161733249-362 Acceptor2 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   4) Thread[id=360, name=qtp161733249-360 Acceptor0 SelectChannelConnector@0.0.0.0:58427, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   5) Thread[id=522, name=qtp2121902167-522 Acceptor1 SelectChannelConnector@0.0.0.0:49081, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   6) Thread[id=524, name=qtp2121902167-524 Acceptor3 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   7) Thread[id=201, name=qtp1313447888-201 Acceptor2 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   8) Thread[id=334, name=qtp1877622224-334 Acceptor2 SelectChannelConnector@0.0.0.0:58424, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   9) Thread[id=348, name=qtp1100289413-348 Acceptor2 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  10) Thread[id=493, name=qtp1376091878-493 Acceptor0 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  11) Thread[id=202, name=qtp1313447888-202 Acceptor3 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  12) Thread[id=199, name=qtp1313447888-199 Acceptor0 SelectChannelConnector@0.0.0.0:58411, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  13) Thread[id=495, name=qtp1376091878-495 Acceptor2 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  14) Thread[id=496, name=qtp1376091878-496 Acceptor3 SelectChannelConnector@0.0.0.0:59385, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  15) Thread[id=214, name=qtp1734092677-214 Acceptor2 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  16) Thread[id=215, name=qtp1734092677-215 Acceptor3 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  17) Thread[id=361, name=qtp161733249-361 Acceptor1 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  18) Thread[id=346, name=qtp1100289413-346 Acceptor0 SelectChannelConnector@0.0.0.0:58425, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  19) Thread[id=347, name=qtp1100289413-347 Acceptor1 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  20) Thread[id=332, name=qtp1877622224-332 Acceptor0 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  21) Thread[id=335, name=qtp1877622224-335 Acceptor3 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  22) Thread[id=509, name=qtp1588150803-509 Acceptor2 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  23) Thread[id=349, name=qtp1100289413-349 Acceptor3 SelectChannelConnector@0.0.0.0:58425, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  24) Thread[id=212, name=qtp1734092677-212 Acceptor0 SelectChannelConnector@0.0.0.0:58413, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  25) Thread[id=508, name=qtp1588150803-508 Acceptor1 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  26) Thread[id=363, name=qtp161733249-363 Acceptor3 SelectChannelConnector@0.0.0.0:58427, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  27) Thread[id=535, name=qtp669515876-535 Acceptor1 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  28) Thread[id=507, name=qtp1588150803-507 Acceptor0 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  29) Thread[id=534, name=qtp669515876-534 Acceptor0 SelectChannelConnector@0.0.0.0:39455, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  30) Thread[id=510, name=qtp1588150803-510 Acceptor3 SelectChannelConnector@0.0.0.0:12040, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  31) Thread[id=537, name=qtp669515876-537 Acceptor3 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  32) Thread[id=213, name=qtp1734092677-213 Acceptor1 SelectChannelConnector@0.0.0.0:58413, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  33) Thread[id=333, name=qtp1877622224-333 Acceptor1 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  34) Thread[id=521, name=qtp2121902167-521 Acceptor0 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  35) Thread[id=536, name=qtp669515876-536 Acceptor2 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
  36) Thread[id=494, name=qtp1376091878-494 Acceptor1 SelectChannelConnector@0.0.0.0:59385, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
	at __randomizedtesting.SeedInfo.seed([640759F856731E77]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.servlet.NoCacheHeaderTest

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.servlet.NoCacheHeaderTest:     1) Thread[id=38, name=qtp10664453-38 Acceptor2 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    2) Thread[id=39, name=qtp10664453-39 Acceptor3 SelectChannelConnector@0.0.0.0:58392, state=RUNNABLE, group=TGRP-NoCacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    3) Thread[id=36, name=qtp10664453-36 Acceptor0 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    4) Thread[id=37, name=qtp10664453-37 Acceptor1 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.servlet.NoCacheHeaderTest: 
   1) Thread[id=38, name=qtp10664453-38 Acceptor2 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   2) Thread[id=39, name=qtp10664453-39 Acceptor3 SelectChannelConnector@0.0.0.0:58392, state=RUNNABLE, group=TGRP-NoCacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   3) Thread[id=36, name=qtp10664453-36 Acceptor0 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   4) Thread[id=37, name=qtp10664453-37 Acceptor1 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
	at __randomizedtesting.SeedInfo.seed([640759F856731E77]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.servlet.NoCacheHeaderTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=38, name=qtp10664453-38 Acceptor2 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    2) Thread[id=39, name=qtp10664453-39 Acceptor3 SelectChannelConnector@0.0.0.0:58392, state=RUNNABLE, group=TGRP-NoCacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    3) Thread[id=36, name=qtp10664453-36 Acceptor0 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)    4) Thread[id=37, name=qtp10664453-37 Acceptor1 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:679)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=38, name=qtp10664453-38 Acceptor2 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   2) Thread[id=39, name=qtp10664453-39 Acceptor3 SelectChannelConnector@0.0.0.0:58392, state=RUNNABLE, group=TGRP-NoCacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   3) Thread[id=36, name=qtp10664453-36 Acceptor0 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
   4) Thread[id=37, name=qtp10664453-37 Acceptor1 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:679)
	at __randomizedtesting.SeedInfo.seed([640759F856731E77]:0)




Build Log:
[...truncated 7638 lines...]
[junit4:junit4] Suite: org.apache.solr.servlet.NoCacheHeaderTest
[junit4:junit4]   2> 2 T29 oas.SolrTestCaseJ4.initCore ####initCore
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solrtest-NoCacheHeaderTest-1352067477466
[junit4:junit4]   2> 3 T29 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4:junit4]   2> 593 T29 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 985 T29 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:58392
[junit4:junit4]   2> 1072 T29 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1073 T29 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1073 T29 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1140 T29 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1140 T29 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1141 T29 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1141 T29 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 1142 T29 oasc.CoreContainer.<init> New CoreContainer 1260777033
[junit4:junit4]   2> 1143 T29 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1143 T29 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1184 T29 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1260 T29 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1261 T29 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1263 T29 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1264 T29 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 1319 T29 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1384 T29 oasc.SolrConfig.<init> Loaded SolrConfig: solr/collection1/conf/solrconfig-nocache.xml
[junit4:junit4]   2> 1385 T29 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1445 T29 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2880 T29 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2897 T29 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2901 T29 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3296 T29 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3313 T29 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3319 T29 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3342 T29 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3345 T29 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solrtest-NoCacheHeaderTest-1352067477466/
[junit4:junit4]   2> 3346 T29 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 3347 T29 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solrtest-NoCacheHeaderTest-1352067477466/index/
[junit4:junit4]   2> 3348 T29 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solrtest-NoCacheHeaderTest-1352067477466/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3356 T29 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solrtest-NoCacheHeaderTest-1352067477466/index forceNew:false
[junit4:junit4]   2> 3360 T29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@cc749e4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@34a083f2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3360 T29 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3362 T29 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3370 T29 oass.SolrIndexSearcher.<init> Opening Searcher@c9021e3 main
[junit4:junit4]   2> 3370 T29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3404 T29 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3405 T29 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3406 T29 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3406 T29 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3407 T29 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3408 T29 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3408 T29 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3409 T29 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3410 T29 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3410 T29 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3411 T29 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3412 T29 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3432 T44 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c9021e3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3432 T29 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 3434 T29 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3434 T29 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3435 T29 oas.SolrJettyTestBase.createJetty Jetty Assigned Port#58392
[junit4:junit4]   2> 3445 T29 oas.SolrTestCaseJ4.setUp ###Starting testLastModified
[junit4:junit4]   2> 3479 T29 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@6e9c2192
[junit4:junit4]   2> 5076 T43 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=379 
[junit4:junit4]   2> 5362 T42 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5366 T41 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5371 T43 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5374 T42 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 5379 T41 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5383 T43 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5386 T42 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 5391 T41 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5395 T43 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5395 T29 oas.SolrTestCaseJ4.tearDown ###Ending testLastModified
[junit4:junit4]   2> 5406 T29 oas.SolrTestCaseJ4.setUp ###Starting testEtag
[junit4:junit4]   2> 5410 T41 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5415 T43 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5419 T41 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5424 T42 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5427 T43 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 5432 T41 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5436 T42 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5440 T43 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5444 T41 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5449 T42 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5450 T29 oas.SolrTestCaseJ4.tearDown ###Ending testEtag
[junit4:junit4]   2> 5459 T29 oas.SolrTestCaseJ4.setUp ###Starting testCacheControl
[junit4:junit4]   2> 5463 T43 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5467 T41 C0 REQ [collection1] webapp=/solr path=/select params={qt=standard&q=solr} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5475 T42 C0 REQ [collection1] webapp=/solr path=/select params={q=solr&qt=standard} hits=0 status=0 QTime=0 
[junit4:junit4]   2> 5476 T29 oas.SolrTestCaseJ4.tearDown ###Ending testCacheControl
[junit4:junit4]   2> 5525 T29 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1260777033
[junit4:junit4]   2> 5525 T29 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6e9c2192
[junit4:junit4]   2> 5526 T29 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 5526 T29 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 5527 T29 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 5527 T29 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 5528 T29 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 5529 T29 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 5631 T29 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 5632 T29 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 5644 T28 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 25824 T28 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.servlet.NoCacheHeaderTest: 
[junit4:junit4]   2> 	   1) Thread[id=38, name=qtp10664453-38 Acceptor2 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]   2> 	   2) Thread[id=39, name=qtp10664453-39 Acceptor3 SelectChannelConnector@0.0.0.0:58392, state=RUNNABLE, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]   2> 	   3) Thread[id=36, name=qtp10664453-36 Acceptor0 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]   2> 	   4) Thread[id=37, name=qtp10664453-37 Acceptor1 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]   2> 25825 T28 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=38, name=qtp10664453-38 Acceptor2 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	   2) Thread[id=39, name=qtp10664453-39 Acceptor3 SelectChannelConnector@0.0.0.0:58392, state=RUNNABLE, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	   3) Thread[id=36, name=qtp10664453-36 Acceptor0 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	   4) Thread[id=37, name=qtp10664453-37 Acceptor1 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 28869 T28 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=38, name=qtp10664453-38 Acceptor2 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]   2> 	   2) Thread[id=39, name=qtp10664453-39 Acceptor3 SelectChannelConnector@0.0.0.0:58392, state=RUNNABLE, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]   2> 	   3) Thread[id=36, name=qtp10664453-36 Acceptor0 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]   2> 	   4) Thread[id=37, name=qtp10664453-37 Acceptor1 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene3x, sim=DefaultSimilarity, locale=da_DK, timezone=SystemV/PST8
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Sun Microsystems Inc. 1.6.0_32 (64-bit)/cpus=16,threads=5,free=457171984,total=514064384
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestPerFieldSimilarity, TestJmxMonitoredMap, NoCacheHeaderTest]
[junit4:junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=NoCacheHeaderTest -Dtests.seed=640759F856731E77 -Dtests.multiplier=3 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=da_DK -Dtests.timezone=SystemV/PST8 -Dtests.file.encoding=ISO8859-1
[junit4:junit4] ERROR   0.00s J0 | NoCacheHeaderTest (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.servlet.NoCacheHeaderTest: 
[junit4:junit4]    >    1) Thread[id=38, name=qtp10664453-38 Acceptor2 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >    2) Thread[id=39, name=qtp10664453-39 Acceptor3 SelectChannelConnector@0.0.0.0:58392, state=RUNNABLE, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >    3) Thread[id=36, name=qtp10664453-36 Acceptor0 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >    4) Thread[id=37, name=qtp10664453-37 Acceptor1 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([640759F856731E77]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=38, name=qtp10664453-38 Acceptor2 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >    2) Thread[id=39, name=qtp10664453-39 Acceptor3 SelectChannelConnector@0.0.0.0:58392, state=RUNNABLE, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >    3) Thread[id=36, name=qtp10664453-36 Acceptor0 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >    4) Thread[id=37, name=qtp10664453-37 Acceptor1 SelectChannelConnector@0.0.0.0:58392, state=BLOCKED, group=TGRP-NoCacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([640759F856731E77]:0)
[junit4:junit4] Completed on J0 in 29.06s, 3 tests, 2 errors <<< FAILURES!

[...truncated 517 lines...]
[junit4:junit4] Suite: org.apache.solr.TestDistributedGrouping
[junit4:junit4]   2> 4 T192 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 798 T192 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 1237 T192 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:58411
[junit4:junit4]   2> 1270 T192 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1271 T192 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1271 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1307 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1308 T192 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1309 T192 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1309 T192 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 1310 T192 oasc.CoreContainer.<init> New CoreContainer 416565147
[junit4:junit4]   2> 1311 T192 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1311 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1358 T192 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1423 T192 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1424 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1425 T192 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1426 T192 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 1505 T192 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1581 T192 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1582 T192 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1608 T192 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2067 T192 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2078 T192 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2081 T192 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2094 T192 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2098 T192 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2102 T192 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2104 T192 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2106 T192 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.TestDistributedGrouping-1352067506724/control/data/
[junit4:junit4]   2> 2107 T192 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44739f3f
[junit4:junit4]   2> 2109 T192 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.TestDistributedGrouping-1352067506724/control/data/index/
[junit4:junit4]   2> 2110 T192 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.TestDistributedGrouping-1352067506724/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2111 T192 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./org.apache.solr.TestDistributedGrouping-1352067506724/control/data/index forceNew:false
[junit4:junit4]   2> 2115 T192 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6765b2e4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3976d4a4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2116 T192 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2117 T192 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2120 T192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2121 T192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2122 T192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2122 T192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2123 T192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2124 T192 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2124 T192 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2125 T192 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2126 T192 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2127 T192 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2128 T192 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2129 T192 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2129 T192 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2130 T192 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2131 T192 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2131 T192 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2132 T192 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2133 T192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2134 T192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2134 T192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2135 T192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2136 T192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2137 T192 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2138 T192 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2138 T192 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2145 T192 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2148 T192 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2149 T192 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-REPLICATION2.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SCHEMA-MINIMAL.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, CURRENCY.XML, SOLRCONFIG-MERGEPOLICY.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, ELEVATE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-CHARFILTERS.XML, SCHEMA-IB.XML, SOLRCONFIG-QUERYSENDER.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, SCHEMA-REPLICATION1.XML, DA_UTF8.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, HYPHENATION.DTD, SOLRCONFIG-ENABLEPLUGIN.XML, SCHEMA-PHRASESUGGEST.XML, STEMDICT.TXT, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, SCHEMA-REVERSED.XML, STOPTYPES-2.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, OPEN-EXCHANGE-RATES.JSON, SCHEMA-SPATIAL.XML, STOPWITHBOM.TXT, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-SPELLCHECKER.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, SCHEMA-XINCLUDE.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA_CODEC.XML, SCHEMA.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, SOLRCONFIG-FUNCTIONQUERY.XML, STOP-2.TXT, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, SCHEMA-FOLDING.XML, STOPWORDS.TXT, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, FRENCHARTICLES.TXT, BAD-MP-SOLRCONFIG.XML, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-DELPOLICY1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, SOLRCONFIG-NATIVELOCK.XML, XSLT, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-ALTDIRECTORY.XML, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, KEEP-2.TXT, SCHEMA12.XML, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 2152 T192 oass.SolrIndexSearcher.<init> Opening Searcher@243ce552 main
[junit4:junit4]   2> 2152 T192 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 2153 T192 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2153 T192 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2153 T192 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2160 T192 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2214 T192 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2214 T192 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2214 T192 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2215 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2215 T192 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2215 T192 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2216 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2216 T192 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2216 T192 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2217 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2233 T206 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@243ce552 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2237 T192 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2238 T192 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1
[junit4:junit4]   2> 2238 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2284 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2300 T192 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 2302 T192 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:58413
[junit4:junit4]   2> 2303 T192 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2303 T192 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2304 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2327 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2328 T192 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2328 T192 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2328 T192 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2329 T192 oasc.CoreContainer.<init> New CoreContainer 575503309
[junit4:junit4]   2> 2329 T192 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2330 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2347 T192 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2360 T192 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2360 T192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2361 T192 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2362 T192 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2399 T192 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2445 T192 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2445 T192 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2459 T192 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2853 T192 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2863 T192 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2865 T192 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2876 T192 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2880 T192 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2883 T192 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2884 T192 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2885 T192 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.TestDistributedGrouping-1352067506724/shard0/data/
[junit4:junit4]   2> 2885 T192 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44739f3f
[junit4:junit4]   2> 2886 T192 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.TestDistributedGrouping-1352067506724/shard0/data/index/
[junit4:junit4]   2> 2887 T192 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.TestDistributedGrouping-1352067506724/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2888 T192 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./org.apache.solr.TestDistributedGrouping-1352067506724/shard0/data/index forceNew:false
[junit4:junit4]   2> 2890 T192 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@229198ea lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fde06bd),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2890 T192 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2891 T192 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2894 T192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2894 T192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2894 T192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2895 T192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2895 T192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2896 T192 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2896 T192 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2897 T192 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2897 T192 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2898 T192 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2899 T192 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2899 T192 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2900 T192 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2900 T192 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2901 T192 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2901 T192 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2902 T192 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2902 T192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2903 T192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2903 T192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2904 T192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2905 T192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2905 T192 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2906 T192 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2906 T192 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2911 T192 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2915 T192 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2916 T192 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-REPLICATION2.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SCHEMA-MINIMAL.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, CURRENCY.XML, SOLRCONFIG-MERGEPOLICY.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, ELEVATE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-CHARFILTERS.XML, SCHEMA-IB.XML, SOLRCONFIG-QUERYSENDER.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, SCHEMA-REPLICATION1.XML, DA_UTF8.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, HYPHENATION.DTD, SOLRCONFIG-ENABLEPLUGIN.XML, SCHEMA-PHRASESUGGEST.XML, STEMDICT.TXT, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, SCHEMA-REVERSED.XML, STOPTYPES-2.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, OPEN-EXCHANGE-RATES.JSON, SCHEMA-SPATIAL.XML, STOPWITHBOM.TXT, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-SPELLCHECKER.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, SCHEMA-XINCLUDE.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA_CODEC.XML, SCHEMA.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, SOLRCONFIG-FUNCTIONQUERY.XML, STOP-2.TXT, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, SCHEMA-FOLDING.XML, STOPWORDS.TXT, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, FRENCHARTICLES.TXT, BAD-MP-SOLRCONFIG.XML, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-DELPOLICY1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, SOLRCONFIG-NATIVELOCK.XML, XSLT, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-ALTDIRECTORY.XML, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, KEEP-2.TXT, SCHEMA12.XML, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 2918 T192 oass.SolrIndexSearcher.<init> Opening Searcher@3eeb5242 main
[junit4:junit4]   2> 2919 T192 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 2919 T192 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2920 T192 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2920 T192 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2926 T192 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2976 T192 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2977 T192 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2977 T192 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2977 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2978 T192 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2978 T192 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2978 T192 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2979 T192 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2979 T192 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2979 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2995 T220 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3eeb5242 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3001 T192 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 3001 T192 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3001 T192 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3002 T192 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@5e4f1ec
[junit4:junit4]   2> 4100 T205 C5 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4:junit4]   2> 4104 T205 C5 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6765b2e4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3976d4a4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4104 T205 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4105 T205 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 41
[junit4:junit4]   2> ASYNC  NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@7eda2096
[junit4:junit4]   2> 4270 T219 C6 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4:junit4]   2> 4272 T219 C6 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@229198ea lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fde06bd),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4273 T219 C6 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4273 T219 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 3
[junit4:junit4]   2> 4307 T203 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 4308 T203 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6765b2e4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3976d4a4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6765b2e4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3976d4a4),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 4308 T203 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 4310 T203 C5 oass.SolrIndexSearcher.<init> Opening Searcher@4d513b99 main
[junit4:junit4]   2> 4310 T203 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 4311 T203 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 4313 T206 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d513b99 main{StandardDirectoryReader(segments_2:2)}
[junit4:junit4]   2> 4314 T203 C5 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 7
[junit4:junit4]   2> 4318 T218 C6 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 4319 T218 C6 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@229198ea lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fde06bd),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@229198ea lockFactory=org.apache.lucene.store.NativeFSLockFactory@7fde06bd),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 4319 T218 C6 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 4320 T218 C6 oass.SolrIndexSearcher.<init> Opening Searcher@6dbb2d63 main
[junit4:junit4]   2> 4320 T218 C6 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 4321 T218 C6 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 4324 T220 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6dbb2d63 main{StandardDirectoryReader(segments_2:2)}
[junit4:junit4]   2> 4324 T218 C6 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 6
[junit4:junit4]   2> 4507 T205 C5 REQ [collection1] webapp=/solr path=/select params={fl=id,a_si&sort=a_si+asc,+id+asc&q=*:*&group.limit=10&group.field=a_si&distrib=false&group=true&wt=javabin&rows=100&version=2} hits=0 status=0 QTime=157 
[junit4:junit4]   2> 4781 T218 C6 REQ [collection1] webapp=/solr path=/select params={group.distributed.first=true&sort=a_si+asc,+id+asc&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&NOW=1352067511241&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&fl=id&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=66 
[junit4:junit4]   2> 4822 T219 C6 REQ [collection1] webapp=/solr path=/select params={fl=id,a_si&sort=a_si+asc,+id+asc&shards=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.limit=10&group.field=a_si&group=true&wt=javabin&rows=100&version=2} status=0 QTime=300 
[junit4:junit4]   2> 4824 T192 oas.BaseDistributedSearchTestCase.query starting stress...
[junit4:junit4]   2> 4864 T228 C6 REQ [collection1] webapp=/solr path=/select params={group.distributed.first=true&sort=a_si+asc,+id+asc&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&NOW=1352067511574&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&fl=id&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 4864 T229 C6 REQ [collection1] webapp=/solr path=/select params={group.distributed.first=true&sort=a_si+asc,+id+asc&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&NOW=1352067511575&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&fl=id&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 4865 T230 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&fl=id&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511574&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 4866 T218 C6 REQ [collection1] webapp=/solr path=/select params={fl=id,a_si&sort=a_si+asc,+id+asc&shards=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.limit=10&group.field=a_si&group=true&wt=javabin&rows=100&version=2} status=0 QTime=11 
[junit4:junit4]   2> 4866 T216 C6 REQ [collection1] webapp=/solr path=/select params={fl=id,a_si&sort=a_si+asc,+id+asc&shards=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.limit=10&group.field=a_si&group=true&wt=javabin&rows=100&version=2} status=0 QTime=10 
[junit4:junit4]   2> 4866 T219 C6 REQ [collection1] webapp=/solr path=/select params={fl=id,a_si&sort=a_si+asc,+id+asc&shards=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.limit=10&group.field=a_si&group=true&wt=javabin&rows=100&version=2} status=0 QTime=11 
[junit4:junit4]   2> 4877 T216 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&fl=id&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511591&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=1 
[junit4:junit4]   2> 4877 T218 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&fl=id&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511591&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 4877 T219 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&fl=id&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511592&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 4878 T229 C6 REQ [collection1] webapp=/solr path=/select params={fl=id,a_si&sort=a_si+asc,+id+asc&shards=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.limit=10&group.field=a_si&group=true&wt=javabin&rows=100&version=2} status=0 QTime=6 
[junit4:junit4]   2> 4878 T228 C6 REQ [collection1] webapp=/solr path=/select params={fl=id,a_si&sort=a_si+asc,+id+asc&shards=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.limit=10&group.field=a_si&group=true&wt=javabin&rows=100&version=2} status=0 QTime=6 
[junit4:junit4]   2> 4878 T230 C6 REQ [collection1] webapp=/solr path=/select params={fl=id,a_si&sort=a_si+asc,+id+asc&shards=[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.limit=10&group.field=a_si&group=true&wt=javabin&rows=100&version=2} status=0 QTime=5 
[junit4:junit4]   2> 4956 T205 C5 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.limit=10&distrib=false&hl.fl=a_t&wt=javabin&hl=true&rows=100&version=2&fl=id,a_si&q=*:*&group.field=a_si&group=true} hits=0 status=0 QTime=74 
[junit4:junit4]   2> 4964 T218 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&hl.fl=a_t&wt=javabin&hl=false&rows=100&version=2&fl=id&shard.url=[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511680&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 4965 T216 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.limit=10&hl.fl=a_t&wt=javabin&hl=true&rows=100&version=2&fl=id,a_si&shards=[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.field=a_si&group=true} status=0 QTime=4 
[junit4:junit4]   2> 4966 T192 oas.BaseDistributedSearchTestCase.query starting stress...
[junit4:junit4]   2> 4973 T216 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&hl.fl=a_t&wt=javabin&hl=false&rows=100&version=2&fl=id&shard.url=[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511689&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 4974 T218 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&hl.fl=a_t&wt=javabin&hl=false&rows=100&version=2&fl=id&shard.url=[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511689&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=1 
[junit4:junit4]   2> 4974 T230 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&hl.fl=a_t&wt=javabin&hl=false&rows=100&version=2&fl=id&shard.url=[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511688&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=1 
[junit4:junit4]   2> 4974 T229 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.limit=10&hl.fl=a_t&wt=javabin&hl=true&rows=100&version=2&fl=id,a_si&shards=[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.field=a_si&group=true} status=0 QTime=4 
[junit4:junit4]   2> 4975 T219 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.limit=10&hl.fl=a_t&wt=javabin&hl=true&rows=100&version=2&fl=id,a_si&shards=[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.field=a_si&group=true} status=0 QTime=6 
[junit4:junit4]   2> 4975 T228 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.limit=10&hl.fl=a_t&wt=javabin&hl=true&rows=100&version=2&fl=id,a_si&shards=[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.field=a_si&group=true} status=0 QTime=5 
[junit4:junit4]   2> 4981 T229 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&hl.fl=a_t&wt=javabin&hl=false&rows=100&version=2&fl=id&shard.url=[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511696&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 4982 T216 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.limit=10&hl.fl=a_t&wt=javabin&hl=true&rows=100&version=2&fl=id,a_si&shards=[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.field=a_si&group=true} status=0 QTime=5 
[junit4:junit4]   2> 4982 T228 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&hl.fl=a_t&wt=javabin&hl=false&rows=100&version=2&fl=id&shard.url=[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511697&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 4983 T219 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&hl.fl=a_t&wt=javabin&hl=false&rows=100&version=2&fl=id&shard.url=[ff01::083]:33332/solr|127.0.0.1:58413/solr&NOW=1352067511697&start=0&q=*:*&group.field=a_si&group=true&isShard=true} status=0 QTime=1 
[junit4:junit4]   2> 4983 T218 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.limit=10&hl.fl=a_t&wt=javabin&hl=true&rows=100&version=2&fl=id,a_si&shards=[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.field=a_si&group=true} status=0 QTime=5 
[junit4:junit4]   2> 4984 T230 C6 REQ [collection1] webapp=/solr path=/select params={sort=a_si+asc,+id+asc&group.limit=10&hl.fl=a_t&wt=javabin&hl=true&rows=100&version=2&fl=id,a_si&shards=[ff01::083]:33332/solr|127.0.0.1:58413/solr&q=*:*&group.field=a_si&group=true} status=0 QTime=6 
[junit4:junit4]   2> 4988 T203 C5 oasr.UnInvertedField.<init> UnInverted multi-valued field {field=a_t,memSize=4288,tindexSize=0,time=0,phase1=0,nTerms=0,bigTerms=0,termInstances=0,uses=0}
[junit4:junit4]   2> 4988 T203 C5 REQ [collection1] webapp=/solr path=/select params={facet=true&sort=a_si+asc,+id+asc&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&fl=id,a_si&q=*:*&group.field=a_si&group=true&facet.field=a_t} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 5001 T216 C6 REQ [collection1] webapp=/solr path=/select params={facet=false&sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&fl=id&shard.url=127.0.0.1:58413/solr|[ff01::083]:33332/solr&NOW=1352067511716&start=0&q=*:*&group.field=a_si&group=true&facet.field=a_t&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 5047 T228 C6 oasr.UnInvertedField.<init> UnInverted multi-valued field {field=a_t,memSize=4288,tindexSize=0,time=0,phase1=0,nTerms=0,bigTerms=0,termInstances=0,uses=0}
[junit4:junit4]   2> 5047 T228 C6 REQ [collection1] webapp=/solr path=/select params={facet=true&sort=a_si+asc,+id+asc&group.limit=10&distrib=false&group.distributed.second=true&wt=javabin&rows=100&f.a_t.facet.limit=160&version=2&fl=id&shard.url=127.0.0.1:58413/solr|[ff01::083]:33332/solr&NOW=1352067511716&start=0&q=*:*&group.field=a_si&group=true&facet.field=a_t&isShard=true} status=0 QTime=23 
[junit4:junit4]   2> 5072 T229 C6 REQ [collection1] webapp=/solr path=/select params={facet=true&sort=a_si+asc,+id+asc&group.limit=10&wt=javabin&rows=100&version=2&fl=id,a_si&shards=127.0.0.1:58413/solr|[ff01::083]:33332/solr&q=*:*&group.field=a_si&group=true&facet.field=a_t} status=0 QTime=75 
[junit4:junit4]   2> 5073 T192 oas.BaseDistributedSearchTestCase.query starting stress...
[junit4:junit4]   2> 5081 T228 C6 REQ [collection1] webapp=/solr path=/select params={facet=false&sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&fl=id&shard.url=127.0.0.1:58413/solr|[ff01::083]:33332/solr&NOW=1352067511796&start=0&q=*:*&group.field=a_si&group=true&facet.field=a_t&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 5081 T216 C6 REQ [collection1] webapp=/solr path=/select params={facet=false&sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&fl=id&shard.url=127.0.0.1:58413/solr|[ff01::083]:33332/solr&NOW=1352067511796&start=0&q=*:*&group.field=a_si&group=true&facet.field=a_t&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 5081 T229 C6 REQ [collection1] webapp=/solr path=/select params={facet=false&sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&fl=id&shard.url=127.0.0.1:58413/solr|[ff01::083]:33332/solr&NOW=1352067511796&start=0&q=*:*&group.field=a_si&group=true&facet.field=a_t&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 5087 T228 C6 REQ [collection1] webapp=/solr path=/select params={facet=true&sort=a_si+asc,+id+asc&group.limit=10&distrib=false&group.distributed.second=true&wt=javabin&rows=100&f.a_t.facet.limit=160&version=2&fl=id&shard.url=127.0.0.1:58413/solr|[ff01::083]:33332/solr&NOW=1352067511796&start=0&q=*:*&group.field=a_si&group=true&facet.field=a_t&isShard=true} status=0 QTime=1 
[junit4:junit4]   2> 5087 T229 C6 REQ [collection1] webapp=/solr path=/select params={facet=true&sort=a_si+asc,+id+asc&group.limit=10&distrib=false&group.distributed.second=true&wt=javabin&rows=100&f.a_t.facet.limit=160&version=2&fl=id&shard.url=127.0.0.1:58413/solr|[ff01::083]:33332/solr&NOW=1352067511796&start=0&q=*:*&group.field=a_si&group=true&facet.field=a_t&isShard=true} status=0 QTime=1 
[junit4:junit4]   2> 5087 T216 C6 REQ [collection1] webapp=/solr path=/select params={facet=true&sort=a_si+asc,+id+asc&group.limit=10&distrib=false&group.distributed.second=true&wt=javabin&rows=100&f.a_t.facet.limit=160&version=2&fl=id&shard.url=127.0.0.1:58413/solr|[ff01::083]:33332/solr&NOW=1352067511796&start=0&q=*:*&group.field=a_si&group=true&facet.field=a_t&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 5088 T218 C6 REQ [collection1] webapp=/solr path=/select params={facet=true&sort=a_si+asc,+id+asc&group.limit=10&wt=javabin&rows=100&version=2&fl=id,a_si&shards=127.0.0.1:58413/solr|[ff01::083]:33332/solr&q=*:*&group.field=a_si&group=true&facet.field=a_t} status=0 QTime=11 
[junit4:junit4]   2> 5087 T219 C6 REQ [collection1] webapp=/solr path=/select params={facet=true&sort=a_si+asc,+id+asc&group.limit=10&wt=javabin&rows=100&version=2&fl=id,a_si&shards=127.0.0.1:58413/solr|[ff01::083]:33332/solr&q=*:*&group.field=a_si&group=true&facet.field=a_t} status=0 QTime=10 
[junit4:junit4]   2> 5088 T230 C6 REQ [collection1] webapp=/solr path=/select params={facet=true&sort=a_si+asc,+id+asc&group.limit=10&wt=javabin&rows=100&version=2&fl=id,a_si&shards=127.0.0.1:58413/solr|[ff01::083]:33332/solr&q=*:*&group.field=a_si&group=true&facet.field=a_t} status=0 QTime=11 
[junit4:junit4]   2> 5096 T219 C6 REQ [collection1] webapp=/solr path=/select params={facet=false&sort=a_si+asc,+id+asc&group.distributed.first=true&group.limit=10&distrib=false&wt=javabin&rows=100&version=2&fl=id&shard.url=127.0.0.1:58413/solr|[ff01::083]:33332/solr&NOW=1352067511811&start=0&q=*:*&group.field=a_si&group=true&facet.field=a_t&isShard=true} status=0 QTime=0 
[junit4:junit4]   2> 5096 T230 C6 REQ [collection1] we

[...truncated too long message...]

t org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >   27) Thread[id=535, name=qtp669515876-535 Acceptor1 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >   28) Thread[id=507, name=qtp1588150803-507 Acceptor0 SelectChannelConnector@0.0.0.0:12040, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >   29) Thread[id=534, name=qtp669515876-534 Acceptor0 SelectChannelConnector@0.0.0.0:39455, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >   30) Thread[id=510, name=qtp1588150803-510 Acceptor3 SelectChannelConnector@0.0.0.0:12040, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >   31) Thread[id=537, name=qtp669515876-537 Acceptor3 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >   32) Thread[id=213, name=qtp1734092677-213 Acceptor1 SelectChannelConnector@0.0.0.0:58413, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >   33) Thread[id=333, name=qtp1877622224-333 Acceptor1 SelectChannelConnector@0.0.0.0:58424, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >   34) Thread[id=521, name=qtp2121902167-521 Acceptor0 SelectChannelConnector@0.0.0.0:49081, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >   35) Thread[id=536, name=qtp669515876-536 Acceptor2 SelectChannelConnector@0.0.0.0:39455, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:147)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    >   36) Thread[id=494, name=qtp1376091878-494 Acceptor1 SelectChannelConnector@0.0.0.0:59385, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:163)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:679)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([640759F856731E77]:0)
[junit4:junit4] Completed on J1 in 50.83s, 1 test, 2 errors <<< FAILURES!

[...truncated 161 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 842 ignored

Total time: 60 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Mime
View raw message