ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy Setrakyan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-885) NPE in GridMapQueryExecutor under load on changing topology
Date Sat, 09 May 2015 15:54:01 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14536735#comment-14536735 ] 

Dmitriy Setrakyan commented on IGNITE-885:
------------------------------------------

At the same time the {{QueryWords}} instance starts hanging. Here is the thread dump:
{code}
Query results [avg=5.34, min=1, max=1615]
(the,1241)
(and,594)
(to,585)
(a,516)
(of,459)
(she,369)
(said,319)
(in,307)
(it,263)
(was,242)
[08:31:32] Topology snapshot [ver=7, nodes=3, CPUs=4, heap=9.6GB]
2015-05-09 08:32:23
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.25-b02 mixed mode):

"ignite-#85%utility-null%" #119 prio=5 os_prio=31 tid=0x00007f88a191e000 nid=0x11207 waiting on condition [0x000000012967b000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#84%utility-null%" #118 prio=5 os_prio=31 tid=0x00007f88a20ce800 nid=0xbf0f waiting on condition [0x0000000129578000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#83%utility-null%" #117 prio=5 os_prio=31 tid=0x00007f88a2024000 nid=0x940b waiting on condition [0x000000012916c000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#82%utility-null%" #115 prio=5 os_prio=31 tid=0x00007f88a192b000 nid=0x10e03 waiting on condition [0x0000000128a0e000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#81%utility-null%" #114 prio=5 os_prio=31 tid=0x00007f88a4303800 nid=0x2f07 waiting on condition [0x000000012890b000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#80%utility-null%" #113 prio=5 os_prio=31 tid=0x00007f88a4196800 nid=0x3307 waiting on condition [0x000000011ec82000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"preloader-demand-worker-#79%null%" #112 prio=5 os_prio=31 tid=0x00007f88a2241000 nid=0x10c03 waiting on condition [0x0000000128808000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000771aa99f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.poll(GridDhtPartitionDemandPool.java:288)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.access$1100(GridDhtPartitionDemandPool.java:56)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool$DemandWorker.body(GridDhtPartitionDemandPool.java:891)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-demand-worker-#78%null%" #111 prio=5 os_prio=31 tid=0x00007f88a223c800 nid=0x10a03 waiting on condition [0x0000000128705000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000771aa9dc8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.poll(GridDhtPartitionDemandPool.java:288)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.access$1100(GridDhtPartitionDemandPool.java:56)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool$DemandWorker.body(GridDhtPartitionDemandPool.java:891)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-supply-worker-#77%null%" #110 prio=5 os_prio=31 tid=0x00007f88a2cc9800 nid=0x10803 waiting on condition [0x0000000128602000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000771aa9200> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.poll(GridDhtPartitionSupplyPool.java:195)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.access$500(GridDhtPartitionSupplyPool.java:46)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool$SupplyWorker.body(GridDhtPartitionSupplyPool.java:215)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-supply-worker-#76%null%" #109 prio=5 os_prio=31 tid=0x00007f88a207f800 nid=0x10603 waiting on condition [0x00000001284ff000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000771aa9200> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.poll(GridDhtPartitionSupplyPool.java:195)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.access$500(GridDhtPartitionSupplyPool.java:46)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool$SupplyWorker.body(GridDhtPartitionSupplyPool.java:215)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"ttl-cleanup-worker-words-#75%null%" #108 prio=5 os_prio=31 tid=0x00007f88a2cc4000 nid=0x10403 waiting on condition [0x00000001283fc000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:6999)
	at org.apache.ignite.internal.processors.cache.GridCacheTtlManager$CleanupWorker.body(GridCacheTtlManager.java:140)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#74%utility-null%" #107 prio=5 os_prio=31 tid=0x00007f88a30c8000 nid=0x10203 waiting on condition [0x00000001280f9000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#73%utility-null%" #106 prio=5 os_prio=31 tid=0x00007f88a1988000 nid=0x10003 waiting on condition [0x0000000127ff6000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#72%utility-null%" #105 prio=5 os_prio=31 tid=0x00007f88a223a000 nid=0xfe03 waiting on condition [0x0000000127ef3000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#71%utility-null%" #104 prio=5 os_prio=31 tid=0x00007f88a1987000 nid=0xfc03 waiting on condition [0x0000000127df0000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#70%utility-null%" #102 prio=5 os_prio=31 tid=0x00007f88a339c800 nid=0xfa03 waiting on condition [0x0000000127ced000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#69%utility-null%" #101 prio=5 os_prio=31 tid=0x00007f88a339f000 nid=0xf803 waiting on condition [0x0000000127bea000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-metrics-logger" #100 prio=5 os_prio=31 tid=0x00007f88a1985000 nid=0xf603 in Object.wait() [0x0000000127ae7000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.util.TimerThread.mainLoop(Timer.java:552)
	- locked <0x00000007722bb7b8> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:505)

"ignite-starvation-checker" #99 prio=5 os_prio=31 tid=0x00007f88a295b000 nid=0xf403 in Object.wait() [0x00000001279e4000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.util.TimerThread.mainLoop(Timer.java:552)
	- locked <0x00000007722bbe98> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:505)

"pool-2-thread-1" #98 prio=5 os_prio=31 tid=0x00007f88a2231800 nid=0xf203 waiting on condition [0x00000001278e1000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02c4bc0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#68%utility-null%" #97 prio=5 os_prio=31 tid=0x00007f88a222d800 nid=0xf003 waiting on condition [0x00000001277de000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#67%utility-null%" #96 prio=5 os_prio=31 tid=0x00007f88a222c000 nid=0xee03 waiting on condition [0x00000001276db000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#66%marshaller-cache-null%" #95 prio=5 os_prio=31 tid=0x00007f88a222b800 nid=0xec03 waiting on condition [0x00000001275d8000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02083d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#65%utility-null%" #94 prio=5 os_prio=31 tid=0x00007f88a283b800 nid=0xea03 waiting on condition [0x00000001274d5000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"preloader-demand-worker-#64%null%" #93 prio=5 os_prio=31 tid=0x00007f88a3286000 nid=0xe803 waiting on condition [0x00000001273d2000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000771e6b3f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.poll(GridDhtPartitionDemandPool.java:288)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.access$1100(GridDhtPartitionDemandPool.java:56)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool$DemandWorker.body(GridDhtPartitionDemandPool.java:891)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-demand-worker-#63%null%" #92 prio=5 os_prio=31 tid=0x00007f88a2286800 nid=0xe603 waiting on condition [0x00000001272cf000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000771e6b898> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.poll(GridDhtPartitionDemandPool.java:288)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.access$1100(GridDhtPartitionDemandPool.java:56)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool$DemandWorker.body(GridDhtPartitionDemandPool.java:891)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-supply-worker-#62%null%" #91 prio=5 os_prio=31 tid=0x00007f88a3285000 nid=0xe403 waiting on condition [0x00000001271cc000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000771e67bc8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.poll(GridDhtPartitionSupplyPool.java:195)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.access$500(GridDhtPartitionSupplyPool.java:46)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool$SupplyWorker.body(GridDhtPartitionSupplyPool.java:215)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-supply-worker-#61%null%" #90 prio=5 os_prio=31 tid=0x00007f88a3436800 nid=0xe203 waiting on condition [0x00000001270c9000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000771e67bc8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.poll(GridDhtPartitionSupplyPool.java:195)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.access$500(GridDhtPartitionSupplyPool.java:46)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool$SupplyWorker.body(GridDhtPartitionSupplyPool.java:215)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"ttl-cleanup-worker-ignite-marshaller-sys-cache-#60%null%" #89 prio=5 os_prio=31 tid=0x00007f88a2275000 nid=0xe003 waiting on condition [0x0000000126fc6000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:6999)
	at org.apache.ignite.internal.processors.cache.GridCacheTtlManager$CleanupWorker.body(GridCacheTtlManager.java:140)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"ttl-cleanup-worker-ignite-atomics-sys-cache-#59%null%" #88 prio=5 os_prio=31 tid=0x00007f88a182d800 nid=0xde03 waiting on condition [0x0000000126ec3000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:6999)
	at org.apache.ignite.internal.processors.cache.GridCacheTtlManager$CleanupWorker.body(GridCacheTtlManager.java:140)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-demand-worker-#58%null%" #87 prio=5 os_prio=31 tid=0x00007f88a3436000 nid=0xdc03 waiting on condition [0x0000000126dc0000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007722b5b60> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.poll(GridDhtPartitionDemandPool.java:288)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.access$1100(GridDhtPartitionDemandPool.java:56)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool$DemandWorker.body(GridDhtPartitionDemandPool.java:891)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-demand-worker-#57%null%" #86 prio=5 os_prio=31 tid=0x00007f88a2272800 nid=0xda03 waiting on condition [0x0000000126cbd000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c114ea58> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.poll(GridDhtPartitionDemandPool.java:288)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.access$1100(GridDhtPartitionDemandPool.java:56)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool$DemandWorker.body(GridDhtPartitionDemandPool.java:891)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-supply-worker-#56%null%" #85 prio=5 os_prio=31 tid=0x00007f88a2272000 nid=0xd803 waiting on condition [0x0000000126bba000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007722b6a40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.poll(GridDhtPartitionSupplyPool.java:195)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.access$500(GridDhtPartitionSupplyPool.java:46)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool$SupplyWorker.body(GridDhtPartitionSupplyPool.java:215)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-supply-worker-#55%null%" #84 prio=5 os_prio=31 tid=0x00007f88a182d000 nid=0xd603 waiting on condition [0x0000000126ab7000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007722b6a40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.poll(GridDhtPartitionSupplyPool.java:195)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.access$500(GridDhtPartitionSupplyPool.java:46)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool$SupplyWorker.body(GridDhtPartitionSupplyPool.java:215)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-demand-worker-#54%null%" #83 prio=5 os_prio=31 tid=0x00007f88a2085800 nid=0xd403 waiting on condition [0x00000001269b4000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007722b6f10> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.poll(GridDhtPartitionDemandPool.java:288)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.access$1100(GridDhtPartitionDemandPool.java:56)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool$DemandWorker.body(GridDhtPartitionDemandPool.java:891)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-demand-worker-#53%null%" #82 prio=5 os_prio=31 tid=0x00007f88a31c6000 nid=0xd203 waiting on condition [0x00000001268b1000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007722b7420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.poll(GridDhtPartitionDemandPool.java:288)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.access$1100(GridDhtPartitionDemandPool.java:56)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool$DemandWorker.body(GridDhtPartitionDemandPool.java:891)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-supply-worker-#52%null%" #81 prio=5 os_prio=31 tid=0x00007f88a3253000 nid=0xd003 waiting on condition [0x00000001267ae000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007722b9d70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.poll(GridDhtPartitionSupplyPool.java:195)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.access$500(GridDhtPartitionSupplyPool.java:46)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool$SupplyWorker.body(GridDhtPartitionSupplyPool.java:215)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"preloader-supply-worker-#51%null%" #80 prio=5 os_prio=31 tid=0x00007f88a2085000 nid=0xce03 waiting on condition [0x00000001266ab000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007722b9d70> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.poll(GridDhtPartitionSupplyPool.java:195)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool.access$500(GridDhtPartitionSupplyPool.java:46)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplyPool$SupplyWorker.body(GridDhtPartitionSupplyPool.java:215)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"ttl-cleanup-worker-ignite-sys-cache-#50%null%" #79 prio=5 os_prio=31 tid=0x00007f88a1827000 nid=0xcc03 waiting on condition [0x00000001265a8000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:6999)
	at org.apache.ignite.internal.processors.cache.GridCacheTtlManager$CleanupWorker.body(GridCacheTtlManager.java:140)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#49%utility-null%" #78 prio=5 os_prio=31 tid=0x00007f88a2102000 nid=0xca03 waiting on condition [0x00000001264a5000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02081b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"exchange-worker-#48%null%" #77 prio=5 os_prio=31 tid=0x00007f88a2121800 nid=0xc803 waiting on condition [0x00000001263a2000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01760f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.poll(GridCachePartitionExchangeManager.java:910)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1800(GridCachePartitionExchangeManager.java:57)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:978)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"grid-time-server-reader-#47%null%" #76 prio=10 os_prio=31 tid=0x00007f88a2b33000 nid=0xc603 runnable [0x0000000125d07000]
   java.lang.Thread.State: RUNNABLE
	at java.net.PlainDatagramSocketImpl.receive0(Native Method)
	- locked <0x00000006c01ed400> (a java.net.PlainDatagramSocketImpl)
	at java.net.AbstractPlainDatagramSocketImpl.receive(AbstractPlainDatagramSocketImpl.java:144)
	- locked <0x00000006c01ed400> (a java.net.PlainDatagramSocketImpl)
	at java.net.DatagramSocket.receive(DatagramSocket.java:812)
	- locked <0x00000006c0b94750> (a java.net.DatagramPacket)
	- locked <0x00000006c01ed3c0> (a java.net.DatagramSocket)
	at org.apache.ignite.internal.processors.clock.GridClockServer$ReadWorker.body(GridClockServer.java:184)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"disco-event-worker-#46%null%" #75 prio=5 os_prio=31 tid=0x00007f88a2b28000 nid=0xc403 waiting on condition [0x0000000125c04000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c0466bc0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:1653)
	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:1636)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"tcp-disco-status-check-sender-#9%null" #74 prio=10 os_prio=31 tid=0x00007f88a3324000 nid=0xc203 waiting on condition [0x0000000125b01000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi$CheckStatusSender.body(TcpDiscoverySpi.java:2340)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

"tcp-disco-hb-sender-#8%null" #73 prio=10 os_prio=31 tid=0x00007f88a2b17000 nid=0xc003 waiting on condition [0x00000001259fe000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi$HeartbeatsSender.body(TcpDiscoverySpi.java:2281)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

"tcp-disco-sock-reader-#7%null" #70 prio=10 os_prio=31 tid=0x00007f88a3320800 nid=0xbc0b runnable [0x00000001255f8000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
	- locked <0x00000006c0017180> (a java.io.BufferedInputStream)
	at org.apache.ignite.marshaller.jdk.JdkMarshallerInputStreamWrapper.read(JdkMarshallerInputStreamWrapper.java:52)
	at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2313)
	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2326)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2797)
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:802)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299)
	at org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.<init>(JdkMarshallerObjectInputStream.java:37)
	at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal(JdkMarshaller.java:96)
	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi$SocketReader.body(TcpDiscoverySpi.java:4807)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

"tcp-disco-srvr-#6%null" #67 prio=10 os_prio=31 tid=0x00007f88a2af5000 nid=0xba03 runnable [0x00000001254f5000]
   java.lang.Thread.State: RUNNABLE
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404)
	at java.net.ServerSocket.implAccept(ServerSocket.java:545)
	at java.net.ServerSocket.accept(ServerSocket.java:513)
	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi$TcpServer.body(TcpDiscoverySpi.java:4571)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

"tcp-disco-msg-worker-#5%null" #66 prio=10 os_prio=31 tid=0x00007f88a31fb000 nid=0xb803 waiting on condition [0x00000001253f2000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c0467d00> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
	at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpiAdapter$MessageWorkerAdapter.body(TcpDiscoverySpiAdapter.java:982)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

"tcp-disco-sock-timeout-worker-#4%null" #65 prio=10 os_prio=31 tid=0x00007f88a31fa000 nid=0xb603 in Object.wait() [0x00000001252ef000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpiAdapter$SocketTimeoutWorker.body(TcpDiscoverySpiAdapter.java:875)
	- locked <0x00000006c00247e0> (a java.lang.Object)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

"metrics-updater-#45%null%" #64 prio=5 os_prio=31 tid=0x00007f88a2ab0800 nid=0xb403 waiting on condition [0x00000001251ec000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.ignite.internal.util.IgniteUtils.sleep(IgniteUtils.java:6999)
	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$MetricsUpdater.body(GridDiscoveryManager.java:1853)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"grid-data-loader-flusher-#44%null%" #63 prio=5 os_prio=31 tid=0x00007f88a2016000 nid=0xb203 waiting on condition [0x00000001250e9000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c0024b68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.DelayQueue.take(DelayQueue.java:211)
	at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$2.body(DataStreamProcessor.java:85)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"grid-nio-worker-3-#43%null%" #62 prio=5 os_prio=31 tid=0x00007f88a21ff800 nid=0xb003 runnable [0x0000000124fe6000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x00000006c0024df8> (a sun.nio.ch.Util$2)
	- locked <0x00000006c0024e08> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000006c0024da8> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1273)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1159)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"grid-nio-worker-2-#42%null%" #61 prio=5 os_prio=31 tid=0x00007f88a326f000 nid=0xae03 runnable [0x0000000124ee3000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x00000006c00263f8> (a sun.nio.ch.Util$2)
	- locked <0x00000006c00263e8> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000006c00262b8> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1273)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1159)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"grid-nio-worker-1-#41%null%" #60 prio=5 os_prio=31 tid=0x00007f88a214c800 nid=0xac03 runnable [0x0000000124de0000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x00000006c00293e8> (a sun.nio.ch.Util$2)
	- locked <0x00000006c00293d8> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000006c00292a8> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1273)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1159)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"grid-nio-worker-0-#40%null%" #59 prio=5 os_prio=31 tid=0x00007f88a31f5000 nid=0xaa03 runnable [0x0000000124cdd000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x00000006c002b930> (a sun.nio.ch.Util$2)
	- locked <0x00000006c002b920> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000006c002b7f0> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1273)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1159)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"nio-acceptor-#39%null%" #58 prio=5 os_prio=31 tid=0x00007f88a2149800 nid=0x300b runnable [0x0000000124bda000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x00000006c0025d38> (a sun.nio.ch.Util$2)
	- locked <0x00000006c0025d28> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000006c0025900> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:1651)
	at org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:1623)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"cron4j::scheduler[94a5f8a813526e592235eaab0000014d394d0a3357650c4b]::timer[94a5f8a813526e592235eaab0000014d394d0a35112fa639]" #57 prio=5 os_prio=31 tid=0x00007f88a19c5000 nid=0xa703 waiting on condition [0x0000000124ad7000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at it.sauronsoftware.cron4j.TimerThread.safeSleep(Unknown Source)
	at it.sauronsoftware.cron4j.TimerThread.run(Unknown Source)

"tcp-comm-sock-timeout-worker-#3%null" #56 prio=5 os_prio=31 tid=0x00007f88a2990800 nid=0xa503 in Object.wait() [0x00000001247d4000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$SocketTimeoutWorker.body(TcpCommunicationSpi.java:2412)
	- locked <0x00000006c00c4a70> (a java.lang.Object)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

"tcp-comm-recovery-worker-#2%null" #55 prio=5 os_prio=31 tid=0x00007f88a3170000 nid=0xa303 waiting on condition [0x00000001246d1000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c00c4ca0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$RecoveryWorker.body(TcpCommunicationSpi.java:2439)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

"nio-idle-client-collector-#1%null" #54 prio=5 os_prio=31 tid=0x00007f88a2ad8800 nid=0xa103 waiting on condition [0x00000001245ce000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$IdleClientWorker.body(TcpCommunicationSpi.java:2232)
	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

"grid-nio-worker-3-#38%null%" #50 prio=5 os_prio=31 tid=0x00007f88a316f800 nid=0x9f03 runnable [0x00000001244cb000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x00000006c00c28e0> (a sun.nio.ch.Util$2)
	- locked <0x00000006c00c28d0> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000006c00c27a0> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1273)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1159)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"grid-nio-worker-2-#37%null%" #49 prio=5 os_prio=31 tid=0x00007f88a221e000 nid=0x9d03 runnable [0x00000001243c8000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x00000006c00c35e8> (a sun.nio.ch.Util$2)
	- locked <0x00000006c00c35d8> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000006c00c34a8> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1273)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1159)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"grid-nio-worker-1-#36%null%" #48 prio=5 os_prio=31 tid=0x00007f88a19ae800 nid=0x9c07 runnable [0x00000001242c5000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x00000006c00c3af0> (a sun.nio.ch.Util$2)
	- locked <0x00000006c00c3ae0> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000006c00c39b0> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1273)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1159)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"grid-nio-worker-0-#35%null%" #47 prio=5 os_prio=31 tid=0x00007f88a3301800 nid=0x980b runnable [0x00000001241c2000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x00000006c00c3ff8> (a sun.nio.ch.Util$2)
	- locked <0x00000006c00c3fe8> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000006c00c3eb8> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:1273)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1159)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"nio-acceptor-#34%null%" #46 prio=5 os_prio=31 tid=0x00007f88a19b0000 nid=0x9907 runnable [0x00000001240bf000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
	- locked <0x00000006c00c3028> (a sun.nio.ch.Util$2)
	- locked <0x00000006c00c3018> (a java.util.Collections$UnmodifiableSet)
	- locked <0x00000006c00c2c00> (a sun.nio.ch.KQueueSelectorImpl)
	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
	at org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:1651)
	at org.apache.ignite.internal.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:1623)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"grid-timeout-worker-#33%null%" #45 prio=5 os_prio=31 tid=0x00007f88a19c9800 nid=0x9103 in Object.wait() [0x0000000123fbc000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:163)
	- locked <0x00000006c0265938> (a java.lang.Object)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
	at java.lang.Thread.run(Thread.java:745)

"ignite-update-notifier-timer" #44 prio=5 os_prio=31 tid=0x00007f88a19ae000 nid=0x8f03 in Object.wait() [0x0000000123cb9000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.util.TimerThread.mainLoop(Timer.java:552)
	- locked <0x00000006c0275558> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:505)

"ignite-clock" #43 daemon prio=10 os_prio=31 tid=0x00007f88a2ac8800 nid=0x8d03 waiting on condition [0x0000000123bb6000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.apache.ignite.internal.util.IgniteUtils$19.run(IgniteUtils.java:2833)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#32%sys-null%" #42 prio=5 os_prio=31 tid=0x00007f88a2abe800 nid=0x8b03 waiting on condition [0x0000000123ab3000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#31%sys-null%" #41 prio=5 os_prio=31 tid=0x00007f88a2215000 nid=0x8903 waiting on condition [0x00000001239b0000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#30%sys-null%" #40 prio=5 os_prio=31 tid=0x00007f88a3252800 nid=0x8703 waiting on condition [0x00000001238ad000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#29%sys-null%" #39 prio=5 os_prio=31 tid=0x00007f88a2214800 nid=0x8503 waiting on condition [0x00000001237aa000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#28%sys-null%" #38 prio=5 os_prio=31 tid=0x00007f88a2abe000 nid=0x8303 waiting on condition [0x00000001236a7000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#27%sys-null%" #37 prio=5 os_prio=31 tid=0x00007f88a19a7800 nid=0x8103 waiting on condition [0x00000001235a4000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#26%sys-null%" #36 prio=5 os_prio=31 tid=0x00007f88a3251800 nid=0x7f03 waiting on condition [0x00000001234a1000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#25%sys-null%" #35 prio=5 os_prio=31 tid=0x00007f88a21db800 nid=0x7d03 waiting on condition [0x000000012339e000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#24%sys-null%" #34 prio=5 os_prio=31 tid=0x00007f88a2abd000 nid=0x7b03 waiting on condition [0x000000012329b000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#23%sys-null%" #33 prio=5 os_prio=31 tid=0x00007f88a19a7000 nid=0x7903 waiting on condition [0x0000000123198000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#22%sys-null%" #32 prio=5 os_prio=31 tid=0x00007f88a2abc800 nid=0x7703 waiting on condition [0x0000000123095000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#21%sys-null%" #31 prio=5 os_prio=31 tid=0x00007f88a19a6000 nid=0x7503 waiting on condition [0x0000000122f92000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#20%sys-null%" #30 prio=5 os_prio=31 tid=0x00007f88a2abb800 nid=0x7303 waiting on condition [0x0000000122e8f000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#19%sys-null%" #29 prio=5 os_prio=31 tid=0x00007f88a2a59000 nid=0x7103 waiting on condition [0x0000000122d8c000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#18%sys-null%" #28 prio=5 os_prio=31 tid=0x00007f88a21db000 nid=0x6f03 waiting on condition [0x0000000122c89000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#17%sys-null%" #27 prio=5 os_prio=31 tid=0x00007f88a19a5800 nid=0x6d03 waiting on condition [0x0000000122b86000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c02052d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#16%pub-null%" #26 prio=5 os_prio=31 tid=0x00007f88a3251000 nid=0x6b03 waiting on condition [0x0000000122a83000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#15%pub-null%" #25 prio=5 os_prio=31 tid=0x00007f88a19a4800 nid=0x6903 waiting on condition [0x0000000122980000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#14%pub-null%" #24 prio=5 os_prio=31 tid=0x00007f88a2a58000 nid=0x6703 waiting on condition [0x000000012287d000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#13%pub-null%" #23 prio=5 os_prio=31 tid=0x00007f88a19a4000 nid=0x6503 waiting on condition [0x000000012277a000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#12%pub-null%" #22 prio=5 os_prio=31 tid=0x00007f88a3250000 nid=0x6303 waiting on condition [0x0000000122677000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#11%pub-null%" #21 prio=5 os_prio=31 tid=0x00007f88a21da000 nid=0x6103 waiting on condition [0x0000000122574000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#10%pub-null%" #20 prio=5 os_prio=31 tid=0x00007f88a19a3000 nid=0x5f03 waiting on condition [0x0000000122471000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#9%pub-null%" #19 prio=5 os_prio=31 tid=0x00007f88a2167800 nid=0x5d03 waiting on condition [0x000000012236e000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#8%pub-null%" #18 prio=5 os_prio=31 tid=0x00007f88a2a57800 nid=0x5b03 waiting on condition [0x000000012226b000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#7%pub-null%" #17 prio=5 os_prio=31 tid=0x00007f88a2167000 nid=0x5903 waiting on condition [0x0000000122168000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#6%pub-null%" #16 prio=5 os_prio=31 tid=0x00007f88a19a2800 nid=0x5703 waiting on condition [0x0000000122065000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#5%pub-null%" #15 prio=5 os_prio=31 tid=0x00007f88a21df000 nid=0x5503 waiting on condition [0x0000000121f62000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#4%pub-null%" #14 prio=5 os_prio=31 tid=0x00007f88a324f800 nid=0x5303 waiting on condition [0x0000000121e5f000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#3%pub-null%" #13 prio=5 os_prio=31 tid=0x00007f88a216c000 nid=0x5103 waiting on condition [0x0000000121d5c000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#2%pub-null%" #12 prio=5 os_prio=31 tid=0x00007f88a324e800 nid=0x4f03 waiting on condition [0x0000000121c59000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"ignite-#1%pub-null%" #11 prio=5 os_prio=31 tid=0x00007f88a2a76800 nid=0x4d03 waiting on condition [0x0000000121b56000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000006c01f2430> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"Monitor Ctrl-Break" #9 daemon prio=5 os_prio=31 tid=0x00007f88a2089000 nid=0x4b03 runnable [0x0000000120e54000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
	- locked <0x000000076f9083b0> (a java.io.InputStreamReader)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.fill(BufferedReader.java:161)
	at java.io.BufferedReader.readLine(BufferedReader.java:324)
	- locked <0x000000076f9083b0> (a java.io.InputStreamReader)
	at java.io.BufferedReader.readLine(BufferedReader.java:389)
	at com.intellij.rt.execution.application.AppMain$1.run(AppMain.java:93)
	at java.lang.Thread.run(Thread.java:745)

"Service Thread" #8 daemon prio=9 os_prio=31 tid=0x00007f88a181c800 nid=0x4703 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C1 CompilerThread2" #7 daemon prio=9 os_prio=31 tid=0x00007f88a181a000 nid=0x4503 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" #6 daemon prio=9 os_prio=31 tid=0x00007f88a1819000 nid=0x4303 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=31 tid=0x00007f88a2020800 nid=0x4103 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=31 tid=0x00007f88a201b000 nid=0x350f waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=31 tid=0x00007f88a280a000 nid=0x2b03 in Object.wait() [0x000000011eb46000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142)
	- locked <0x00000006c02d3288> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158)
	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" #2 daemon prio=10 os_prio=31 tid=0x00007f88a3013000 nid=0x2903 in Object.wait() [0x000000011ea43000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:502)
	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
	- locked <0x00000006c034f8a8> (a java.lang.ref.Reference$Lock)

"main" #1 prio=5 os_prio=31 tid=0x00007f88a2800800 nid=0xd03 waiting on condition [0x0000000103784000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x000000076d19f3c0> (a java.util.concurrent.CountDownLatch$Sync)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
	at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
	at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.query(GridReduceQueryExecutor.java:340)
	at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryTwoStep(IgniteH2Indexing.java:775)
	at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.queryTwoStep(IgniteH2Indexing.java:866)
	at org.apache.ignite.internal.processors.query.GridQueryProcessor.queryTwoStep(GridQueryProcessor.java:561)
	at org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:513)
	at org.apache.ignite.examples.streaming.wordcount.QueryWords.main(QueryWords.java:75)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

"VM Thread" os_prio=31 tid=0x00007f88a300e000 nid=0x2703 runnable 

"GC task thread#0 (ParallelGC)" os_prio=31 tid=0x00007f88a280d800 nid=0x1803 runnable 

"GC task thread#1 (ParallelGC)" os_prio=31 tid=0x00007f88a280e000 nid=0x1a03 runnable 

"GC task thread#2 (ParallelGC)" os_prio=31 tid=0x00007f88a280e800 nid=0x1c03 runnable 

"GC task thread#3 (ParallelGC)" os_prio=31 tid=0x00007f88a280f800 nid=0x1e03 runnable 

"VM Periodic Task Thread" os_prio=31 tid=0x00007f88a1813000 nid=0x4903 waiting on condition 

JNI global references: 251

Heap
 PSYoungGen      total 109568K, used 91629K [0x000000076ab00000, 0x0000000772400000, 0x00000007c0000000)
  eden space 98816K, 81% used [0x000000076ab00000,0x000000076f9ff408,0x0000000770b80000)
  from space 10752K, 99% used [0x0000000771980000,0x00000007723fc040,0x0000000772400000)
  to   space 10752K, 0% used [0x0000000770f00000,0x0000000770f00000,0x0000000771980000)
 ParOldGen       total 112640K, used 32356K [0x00000006c0000000, 0x00000006c6e00000, 0x000000076ab00000)
  object space 112640K, 28% used [0x00000006c0000000,0x00000006c1f993c8,0x00000006c6e00000)
 Metaspace       used 30436K, capacity 30776K, committed 31104K, reserved 1077248K
  class space    used 3346K, capacity 3440K, committed 3456K, reserved 1048576K
{code}

> NPE in GridMapQueryExecutor under load on changing topology
> -----------------------------------------------------------
>
>                 Key: IGNITE-885
>                 URL: https://issues.apache.org/jira/browse/IGNITE-885
>             Project: Ignite
>          Issue Type: Bug
>          Components: cache
>            Reporter: Dmitriy Setrakyan
>            Assignee: Sergi Vladykin
>            Priority: Blocker
>             Fix For: sprint-4
>
>
> The error is reproduced with "streaming.wordcount" example.
> # Start instance of {{ExampleNodeStartup}}
> # Start {{StreamWords}} instance
> # Start {{QueryWords}} instance
> # Watch query results coming
> # Start another {{ExampleNodeStartup}} instance.
> On the new {{ExampleNodeStartup}} I get the following error:
> {code}
> [08:31:31]   /  _/ ___/ |/ /  _/_  __/ __/ 
> [08:31:31]  _/ // (7 7    // /  / / / _/   
> [08:31:31] /___/\___/_/|_/___/ /_/ /___/  
> [08:31:31]  
> [08:31:31] ver. 1.0.4#20150507-sha1:0c13a08b
> [08:31:31] 2015 Copyright(C) Apache Software Foundation
> [08:31:31] 
> [08:31:31] Quiet mode.
> [08:31:31]   ^-- Logging to file '/Users/Dmitriy/GridGain/gitroot/incubator-ignite/work/log/ignite-0b843b9b.log'
> [08:31:31]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
> [08:31:31] 
> [08:31:31] Configured plugins:
> [08:31:31]   ^-- None
> [08:31:31] 
> [08:31:33,018][ERROR][ignite-#8%pub-null%][GridMapQueryExecutor] Failed to process message: GridQueryRequest [reqId=57, pageSize=1024, space=words, qrys=[GridCacheSqlQuery [alias=__Z0(), qry=SELECT
>     _VAL __C0,
>     COUNT(_VAL) __C1
> FROM "words".STRING
> GROUP BY __C0
> ORDER BY 2 DESC LIMIT 10, params=[]]]]
> java.lang.NullPointerException
> 	at org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest(GridMapQueryExecutor.java:225)
> 	at org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onMessage(GridMapQueryExecutor.java:150)
> 	at org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor$2.onMessage(GridMapQueryExecutor.java:125)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager$ArrayListener.onMessage(GridIoManager.java:1603)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:690)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager.access$1500(GridIoManager.java:58)
> 	at org.apache.ignite.internal.managers.communication.GridIoManager$5.run(GridIoManager.java:653)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> [08:31:34] Performance suggestions for grid  (fix if possible)
> [08:31:34] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
> [08:31:34]   ^-- Disable peer class loading (set 'peerClassLoadingEnabled' to false)
> [08:31:34]   ^-- Disable grid events (remove 'includeEventTypes' from configuration)
> [08:31:34] 
> [08:31:34] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
> [08:31:34] 
> [08:31:34] Ignite node started OK (id=0b843b9b)
> [08:31:34] Topology snapshot [ver=7, nodes=3, CPUs=4, heap=9.6GB]
> ...
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message