ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lukas Lentner <kont...@lukaslentner.de>
Subject NullPointer in GridAffinityAssignment.initPrimaryBackupMaps
Date Mon, 13 Nov 2017 10:31:19 GMT
Hi,

I tried to start an additional node in my cluster and I got the following error. Can you give me a reason for that? What can I do to circumvent such problems?

2017-11-11 06:10:34:969 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal -
>>> __________ ________________
>>> / _/ ___/ |/ / _/_ __/ __/
>>> _/ // (7 7 // / / / / _/
>>> /___/\___/_/|_/___/ /_/ /___/
>>>
>>> ver. 2.3.0#20171028-sha1:8add7fd5
>>> 2017 Copyright(C) Apache Software Foundation
>>>
>>> Ignite documentation: http://ignite.apache.org
2017-11-11 06:10:34:969 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Config URL: n/a
2017-11-11 06:10:34:969 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Daemon mode: off
2017-11-11 06:10:34:969 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - OS: Linux 4.9.32-15.41.amzn1.x86_64 amd64
2017-11-11 06:10:34:969 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - OS user: root
2017-11-11 06:10:34:970 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - PID: 8
2017-11-11 06:10:34:970 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Language runtime: Java Platform API Specification ver. 1.8
2017-11-11 06:10:34:970 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - VM information: OpenJDK Runtime Environment 1.8.0_151-b12 Oracle Corporation OpenJDK 64-Bit Server VM 25.151-b12
2017-11-11 06:10:34:971 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - VM total memory: 12.0GB
2017-11-11 06:10:34:971 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Remote Management [restart: off, REST: on, JMX (remote: off)]
2017-11-11 06:10:34:973 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - IGNITE_HOME=null
2017-11-11 06:10:34:973 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - VM arguments: [-XX:+UseG1GC, -XX:+AlwaysPreTouch, -XX:+ScavengeBeforeFullGC, -XX:+DisableExplicitGC, -Xms12G, -Xmx12G, -XX:MetaspaceSize=2G, -XX:MaxMetaspaceSize=2G, -Djava.net.preferIPv4Stack=true, -Djava.awt.headless=true, -agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n]
2017-11-11 06:10:34:974 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - System cache's DataRegion size is configured to 40 MB. Use DataStorageConfiguration.systemCacheMemorySize property to change the setting.
2017-11-11 06:10:34:980 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Configured caches [in 'sysMemPlc' dataRegion: ['ignite-sys-cache']]
2017-11-11 06:10:34:983 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Local node user attribute [xxxxxx.xxxs.domain.grid.globalUserAttribute.AppIdUserAttribute=xxx.xxx.v3.xxx]
2017-11-11 06:10:34:983 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Local node user attribute [xxxxxx.xxxs.domain.grid.globalUserAttribute.AppVersionUserAttribute=423]
2017-11-11 06:10:34:983 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Local node user attribute [xxxxxx.xxxs.domain.grid.globalUserAttribute.AppProjectIdUserAttribute=xxx.xxx.v3]
2017-11-11 06:10:34:984 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Local node user attribute [xxxxxx.xxxs.domain.grid.globalUserAttribute.RoleUserAttribute=PRIMARY_WORKER]
2017-11-11 06:10:34:984 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Local node user attribute [xxxxxx.xxxs.domain.grid.globalUserAttribute.NetworkExternalAddressUserAttribute=10.10.10.146]
2017-11-11 06:10:34:984 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Local node user attribute [xxxxxx.xxxs.domain.grid.globalUserAttribute.NetworkDiscoveryPortUserAttribute=31000]
2017-11-11 06:10:34:984 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Local node user attribute [xxxxxx.xxxs.domain.grid.globalUserAttribute.NetworkCommunicationPortUserAttribute=30000]
2017-11-11 06:10:34:984 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Local node user attribute [xxxxxx.xxx.xxx.v3.domain.grid.userAttribute.xxx=true]
2017-11-11 06:10:35:025 +0000 [main] INFO org.apache.ignite.internal.processors.plugin.IgnitePluginProcessor - Configured plugins:
2017-11-11 06:10:35:025 +0000 [main] INFO org.apache.ignite.internal.processors.plugin.IgnitePluginProcessor - ^-- None
2017-11-11 06:10:35:025 +0000 [main] INFO org.apache.ignite.internal.processors.plugin.IgnitePluginProcessor -
2017-11-11 06:10:35:071 +0000 [main] INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi - Successfully bound communication NIO server to TCP port [port=30000, locHost=0.0.0.0/0.0.0.0, selectorsCnt=4, selectorSpins=0, pairedConn=false]
2017-11-11 06:10:35:074 +0000 [main] INFO xxxxxx.xxxs.domain.grid.DockerAddressResolver - Resolving /172.17.0.2:30000 to ip-10-10-10-146.eu-central-1.compute.internal/10.10.10.146:30000
2017-11-11 06:10:35:074 +0000 [main] INFO xxxxxx.xxxs.domain.grid.DockerAddressResolver - Not resolving /127.0.0.1:30000
2017-11-11 06:10:35:074 +0000 [main] INFO xxxxxx.xxxs.domain.grid.DockerAddressResolver - Resolving 8a48b3b99a6e/172.17.0.2:30000 to ip-10-10-10-146.eu-central-1.compute.internal/10.10.10.146:30000
2017-11-11 06:10:35:074 +0000 [main] WARN org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi - Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.
2017-11-11 06:10:35:085 +0000 [main] WARN org.apache.ignite.spi.checkpoint.noop.NoopCheckpointSpi - Checkpoints are disabled (to enable configure any GridCheckpointSpi implementation)
2017-11-11 06:10:35:107 +0000 [main] WARN org.apache.ignite.internal.managers.collision.GridCollisionManager - Collision resolution is disabled (all jobs will be activated upon arrival).
2017-11-11 06:10:35:108 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Security status [authentication=off, tls/ssl=off]
2017-11-11 06:10:35:298 +0000 [main] INFO org.apache.ignite.internal.processors.odbc.ClientListenerProcessor - Client connector processor has started on TCP port 10800
2017-11-11 06:10:35:336 +0000 [main] INFO org.apache.ignite.internal.processors.rest.protocols.tcp.GridTcpRestProtocol - Command protocol successfully started [name=TCP binary, host=0.0.0.0/0.0.0.0, port=11211]
2017-11-11 06:10:35:368 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Non-loopback local IPs: 172.17.0.2
2017-11-11 06:10:35:368 +0000 [main] INFO org.apache.ignite.internal.IgniteKernal - Enabled local MACs: 0242AC110002
2017-11-11 06:10:35:407 +0000 [main] INFO org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Successfully bound to TCP port [port=31000, localHost=0.0.0.0/0.0.0.0, locNodeId=2e01c750-0ae2-408b-8822-99aafc4ee434]
2017-11-11 06:10:35:410 +0000 [main] INFO xxxxxx.xxxs.domain.grid.DockerAddressResolver - Resolving /172.17.0.2:31000 to ip-10-10-10-146.eu-central-1.compute.internal/10.10.10.146:31000
2017-11-11 06:10:35:411 +0000 [main] INFO xxxxxx.xxxs.domain.grid.DockerAddressResolver - Not resolving /127.0.0.1:31000
2017-11-11 06:10:35:411 +0000 [main] INFO xxxxxx.xxxs.domain.grid.DockerAddressResolver - Resolving 8a48b3b99a6e/172.17.0.2:31000 to ip-10-10-10-146.eu-central-1.compute.internal/10.10.10.146:31000
2017-11-11 06:10:35:411 +0000 [main] INFO xxxxxx.xxxs.domain.grid.DockerAddressResolver - Not resolving /127.0.0.1:31000
2017-11-11 06:10:35:411 +0000 [main] INFO xxxxxx.xxxs.domain.grid.DockerAddressResolver - Resolving 8a48b3b99a6e/172.17.0.2:31000 to ip-10-10-10-146.eu-central-1.compute.internal/10.10.10.146:31000
Nov 11, 2017 6:10:35 AM java.util.logging.LogManager$RootLogger log
SEVERE: Failed to resolve default logging config file: config/java.util.logging.properties
2017-11-11 06:10:35:774 +0000 [tcp-disco-srvr-#3] INFO org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - TCP discovery accepted incoming connection [rmtAddr=/10.10.1.211, rmtPort=47565]
2017-11-11 06:10:35:780 +0000 [tcp-disco-srvr-#3] INFO org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - TCP discovery spawning a new thread for connection [rmtAddr=/10.10.1.211, rmtPort=47565]
2017-11-11 06:10:35:780 +0000 [tcp-disco-sock-reader-#4] INFO org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Started serving remote node connection [rmtAddr=/10.10.1.211:47565, rmtPort=47565]
2017-11-11 06:10:36:102 +0000 [main] WARN org.apache.ignite.internal.IgniteKernal - Nodes started on local machine require more than 80% of physical RAM what can lead to significant slowdown due to swapping (please decrease JVM heap size, data region size or checkpoint buffer size) [required=76799MB, available=15041MB]
2017-11-11 06:10:36:124 +0000 [exchange-worker-#42] INFO org.apache.ignite.internal.exchange.time - Started exchange init [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], crd=false, evt=NODE_JOINED, evtNode=2e01c750-0ae2-408b-8822-99aafc4ee434, customEvt=null, allowMerge=true]
2017-11-11 06:10:36:221 +0000 [exchange-worker-#42] INFO org.apache.ignite.internal.processors.cache.GridCacheProcessor - Started cache [name=ignite-sys-cache, memoryPolicyName=sysMemPlc, mode=REPLICATED, atomicity=TRANSACTIONAL]
2017-11-11 06:10:36:230 +0000 [exchange-worker-#42] INFO org.apache.ignite.internal.processors.cache.GridCacheProcessor - Started cache [name=xxxxxx.xxx.xxx.v3.domain.grid.userAttribute.LayoutTaskQueueUserAttribute_423, memoryPolicyName=default, mode=REPLICATED, atomicity=TRANSACTIONAL]
2017-11-11 06:10:36:248 +0000 [exchange-worker-#42] INFO org.apache.ignite.internal.processors.cache.GridCacheProcessor - Started cache [name=xxxxxx.xxx.xxx.v3.domain.grid.userAttribute.LayoutTaskWorkbenchUserAttribute_423, memoryPolicyName=default, mode=REPLICATED, atomicity=TRANSACTIONAL]
2017-11-11 06:10:36:251 +0000 [exchange-worker-#42] INFO org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture - Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], waitTime=0ms, futInfo=NA]
2017-11-11 06:10:36:267 +0000 [grid-nio-worker-tcp-comm-0-#25] INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi - Accepted incoming communication connection [locAddr=/127.0.0.1:30000, rmtAddr=/127.0.0.1:47042]
2017-11-11 06:10:36:472 +0000 [grid-nio-worker-tcp-comm-1-#26] INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi - Accepted incoming communication connection [locAddr=/127.0.0.1:30000, rmtAddr=/127.0.0.1:47044]
2017-11-11 06:10:36:674 +0000 [grid-nio-worker-tcp-comm-2-#27] INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi - Accepted incoming communication connection [locAddr=/127.0.0.1:30000, rmtAddr=/127.0.0.1:47046]
2017-11-11 06:10:36:875 +0000 [grid-nio-worker-tcp-comm-3-#28] INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi - Accepted incoming communication connection [locAddr=/127.0.0.1:30000, rmtAddr=/127.0.0.1:47048]
2017-11-11 06:10:37:076 +0000 [grid-nio-worker-tcp-comm-0-#25] INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi - Accepted incoming communication connection [locAddr=/127.0.0.1:30000, rmtAddr=/127.0.0.1:47050]
2017-11-11 06:10:37:077 +0000 [grid-nio-worker-tcp-comm-1-#26] INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi - Accepted incoming communication connection [locAddr=/172.17.0.2:30000, rmtAddr=/172.17.0.2:53726]
2017-11-11 06:10:37:081 +0000 [grid-nio-worker-tcp-comm-2-#27] INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi - Established outgoing communication connection [locAddr=/172.17.0.2:46354, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000]
2017-11-11 06:10:37:082 +0000 [exchange-worker-#42] INFO org.apache.ignite.internal.exchange.time - Finished exchange init [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], crd=false]
2017-11-11 06:10:37:120 +0000 [sys-#44] INFO org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture - Received full message, will finish exchange [node=c68d7211-41ac-4364-81b5-46f55f62463e, resVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
2017-11-11 06:10:37:126 +0000 [sys-#44] ERROR org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture - Failed to notify listener: o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$5@47cee2c8
java.lang.NullPointerException
at org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps(GridAffinityAssignment.java:248)
at org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.<init>(GridAffinityAssignment.java:95)
at org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.initialize(GridAffinityAssignmentCache.java:190)
at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager$13.applyx(CacheAffinitySharedManager.java:1320)
at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager$13.applyx(CacheAffinitySharedManager.java:1289)
at org.apache.ignite.internal.util.lang.IgniteInClosureX.apply(IgniteInClosureX.java:38)
at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.forAllCacheGroups(CacheAffinitySharedManager.java:1077)
at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onLocalJoin(CacheAffinitySharedManager.java:1289)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processFullMessage(GridDhtPartitionsExchangeFuture.java:2765)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$1400(GridDhtPartitionsExchangeFuture.java:116)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$5.apply(GridDhtPartitionsExchangeFuture.java:2524)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$5.apply(GridDhtPartitionsExchangeFuture.java:2512)
at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:353)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onReceiveFullMessage(GridDhtPartitionsExchangeFuture.java:2512)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processFullPartitionUpdate(GridCachePartitionExchangeManager.java:1434)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1100(GridCachePartitionExchangeManager.java:131)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$3.onMessage(GridCachePartitionExchangeManager.java:334)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$3.onMessage(GridCachePartitionExchangeManager.java:332)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:2627)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:2606)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1060)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293)
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2017-11-11 06:10:37:130 +0000 [sys-#44] ERROR org.apache.ignite.internal.processors.cache.GridCacheIoManager - Failed processing message [senderId=c68d7211-41ac-4364-81b5-46f55f62463e, msg=GridDhtPartitionsFullMessage [parts={-2100569601=GridDhtPartitionFullMap [nodeId=c68d7211-41ac-4364-81b5-46f55f62463e, nodeOrder=1, updateSeq=128, size=6], 1605849125=GridDhtPartitionFullMap [nodeId=c68d7211-41ac-4364-81b5-46f55f62463e, nodeOrder=1, updateSeq=94, size=4], -150435437=GridDhtPartitionFullMap [nodeId=c68d7211-41ac-4364-81b5-46f55f62463e, nodeOrder=1, updateSeq=96, size=4]}, partCntrs=o.a.i.i.processors.cache.distributed.dht.preloader.IgniteDhtPartitionCountersMap@2b84c42e, partCntrs2=o.a.i.i.processors.cache.distributed.dht.preloader.IgniteDhtPartitionCountersMap2@b712c5f, partHistSuppliers=o.a.i.i.processors.cache.distributed.dht.preloader.IgniteDhtPartitionHistorySuppliersMap@220b1b9a, partsToReload=o.a.i.i.processors.cache.distributed.dht.preloader.IgniteDhtPartitionsToReloadMap@7b2630b3, topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], errs=
{}
, compress=false, resTopVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], partCnt=3, super=GridDhtPartitionsAbstractMessage [exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=null, nodeId=2e01c750, evt=NODE_JOINED], lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], super=GridCacheMessage [msgId=276, depInfo=null, err=null, skipPrepare=false]]]]
java.lang.NullPointerException
at org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.initPrimaryBackupMaps(GridAffinityAssignment.java:248)
at org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.<init>(GridAffinityAssignment.java:95)
at org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.initialize(GridAffinityAssignmentCache.java:190)
at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager$13.applyx(CacheAffinitySharedManager.java:1320)
at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager$13.applyx(CacheAffinitySharedManager.java:1289)
at org.apache.ignite.internal.util.lang.IgniteInClosureX.apply(IgniteInClosureX.java:38)
at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.forAllCacheGroups(CacheAffinitySharedManager.java:1077)
at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onLocalJoin(CacheAffinitySharedManager.java:1289)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.processFullMessage(GridDhtPartitionsExchangeFuture.java:2765)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.access$1400(GridDhtPartitionsExchangeFuture.java:116)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$5.apply(GridDhtPartitionsExchangeFuture.java:2524)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture$5.apply(GridDhtPartitionsExchangeFuture.java:2512)
at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:353)
at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onReceiveFullMessage(GridDhtPartitionsExchangeFuture.java:2512)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.processFullPartitionUpdate(GridCachePartitionExchangeManager.java:1434)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.access$1100(GridCachePartitionExchangeManager.java:131)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$3.onMessage(GridCachePartitionExchangeManager.java:334)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$3.onMessage(GridCachePartitionExchangeManager.java:332)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:2627)
at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$MessageHandler.apply(GridCachePartitionExchangeManager.java:2606)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1060)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99)
at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293)
at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2017-11-11 06:10:46:118 +0000 [ignite-update-notifier-timer] INFO org.apache.ignite.internal.processors.cluster.GridUpdateNotifier - Your version is up to date.
2017-11-11 06:10:47:082 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:10:47:083 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Ready affinity version: AffinityTopologyVersion [topVer=-1, minorTopVer=0]
2017-11-11 06:10:47:086 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Last exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380646073, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380646073, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]
2017-11-11 06:10:47:086 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - First 10 pending exchange futures [total=0]
2017-11-11 06:10:47:088 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Last 10 exchange futures (total: 1):
2017-11-11 06:10:47:088 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - >>> GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], evt=NODE_JOINED, evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380646073, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], done=false]
2017-11-11 06:10:47:088 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending transactions:
2017-11-11 06:10:47:088 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending explicit locks:
2017-11-11 06:10:47:088 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending cache futures:
2017-11-11 06:10:47:088 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending atomic cache futures:
2017-11-11 06:10:47:088 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending data streamer futures:
2017-11-11 06:10:47:088 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending transaction deadlock detection futures:
2017-11-11 06:10:47:104 +0000 [sys-#45] INFO org.apache.ignite.internal.diagnostic - Exchange future waiting for coordinator response [crd=c68d7211-41ac-4364-81b5-46f55f62463e, topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
Remote node information:
General node info [id=c68d7211-41ac-4364-81b5-46f55f62463e, client=false, discoTopVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], time=06:10:47.102]
Partitions exchange info [readyVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
Last initialized exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 25374b3146cd/172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380645531, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1280119156], init=true, lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636067, centralizedAff=false, changeGlobalStateE=null, done=true, state=DONE, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=14, minorTopVer=0], hash=2059584827]]
Communication SPI statistics [rmtNode=2e01c750-0ae2-408b-8822-99aafc4ee434]
Communication SPI recovery descriptors:
[key=ConnectionKey [nodeId=2e01c750-0ae2-408b-8822-99aafc4ee434, idx=0, connCnt=0], msgsSent=1, msgsAckedByRmt=0, msgsRcvd=2, lastAcked=0, reserveCnt=1, descIdHash=1607104290]
Communication SPI clients:
[node=2e01c750-0ae2-408b-8822-99aafc4ee434, client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=49953, bytesSent=191751, bytesRcvd0=853, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=null, finished=false, hashCode=497542225, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#26]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=2, sentCnt=1, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=true, connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=2, sentCnt=1, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=true, connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false], super=GridNioSessionImpl [locAddr=/172.17.0.2:30000, rmtAddr=/10.10.10.146:46354, createTime=1510380637080, closeTime=0, bytesSent=47134, bytesRcvd=1317, bytesSent0=0, bytesRcvd0=853, sndSchedTime=1510380637080, lastSndTime=1510380637101, lastRcvTime=1510380647092, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@5a24c5cd, directMode=true], GridConnectionBytesVerifyFilter], accepted=true]], super=GridAbstractCommunicationClient [lastUsed=1510380637080, closed=false, connIdx=0]]]
NIO sessions statistics:
>> Selector info [idx=1, keysCnt=1, bytesRcvd=49953, bytesRcvd0=853, bytesSent=191751, bytesSent0=0]
Connection info [in=true, rmtAddr=/10.10.10.146:46354, locAddr=/172.17.0.2:30000, msgsSent=1, msgsAckedByRmt=0, descIdHash=1607104290, unackedMsgs=[GridDhtPartitionsFullMessage], msgsRcvd=2, lastAcked=0, descIdHash=1607104290, bytesRcvd=1317, bytesRcvd0=853, bytesSent=47134, bytesSent0=0, opQueueSize=0]
Exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 25374b3146cd/172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380645531, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1280119156], init=true, lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636067, centralizedAff=false, changeGlobalStateE=null, done=true, state=DONE, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=14, minorTopVer=0], hash=2059584827]]
Local communication statistics:
Communication SPI statistics [rmtNode=c68d7211-41ac-4364-81b5-46f55f62463e]
Communication SPI recovery descriptors:
[key=ConnectionKey [nodeId=c68d7211-41ac-4364-81b5-46f55f62463e, idx=0, connCnt=-1], msgsSent=2, msgsAckedByRmt=0, msgsRcvd=2, lastAcked=0, reserveCnt=7, descIdHash=980007565]
Communication SPI clients:
[node=c68d7211-41ac-4364-81b5-46f55f62463e, client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=2, bytesRcvd=55876, bytesSent=1293, bytesRcvd0=8770, bytesSent0=853, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=null, finished=false, hashCode=1240667813, interrupted=false, runner=grid-nio-worker-tcp-comm-2-#27]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=2, sentCnt=2, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=7, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=2, sentCnt=2, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=7, pairedConnections=false], super=GridNioSessionImpl [locAddr=/172.17.0.2:46354, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000, createTime=1510380637074, closeTime=0, bytesSent=1275, bytesRcvd=55876, bytesSent0=853, bytesRcvd0=8770, sndSchedTime=1510380637074, lastSndTime=1510380647087, lastRcvTime=1510380647097, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@204cfb1c, directMode=true], GridConnectionBytesVerifyFilter], accepted=false]], super=GridAbstractCommunicationClient [lastUsed=1510380637074, closed=false, connIdx=0]]]
NIO sessions statistics:
>> Selector info [idx=2, keysCnt=1, bytesRcvd=55876, bytesRcvd0=8770, bytesSent=1293, bytesSent0=853]
Connection info [in=false, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000, locAddr=/172.17.0.2:46354, msgsSent=2, msgsAckedByRmt=0, descIdHash=980007565, unackedMsgs=[GridDhtPartitionsSingleMessage, IgniteDiagnosticMessage], msgsRcvd=2, lastAcked=0, descIdHash=980007565, bytesRcvd=55876, bytesRcvd0=8770, bytesSent=1275, bytesSent0=853, opQueueSize=0]
2017-11-11 06:10:56:120 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Failed to wait for initial partition map exchange. Possible reasons are:
^-- Transactions in deadlock.
^-- Long running transactions (ignore if this is the case).
^-- Unreleased explicit locks.
2017-11-11 06:10:57:092 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:10:57:092 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Ready affinity version: AffinityTopologyVersion [topVer=-1, minorTopVer=0]
2017-11-11 06:10:57:093 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Last exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380656075, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380656075, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]
2017-11-11 06:10:57:093 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - First 10 pending exchange futures [total=0]
2017-11-11 06:10:57:093 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Last 10 exchange futures (total: 1):
2017-11-11 06:10:57:093 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - >>> GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], evt=NODE_JOINED, evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380656075, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], done=false]
2017-11-11 06:10:57:094 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending transactions:
2017-11-11 06:10:57:094 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending explicit locks:
2017-11-11 06:10:57:094 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending cache futures:
2017-11-11 06:10:57:094 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending atomic cache futures:
2017-11-11 06:10:57:094 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending data streamer futures:
2017-11-11 06:10:57:094 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending transaction deadlock detection futures:
2017-11-11 06:10:57:098 +0000 [grid-nio-worker-tcp-comm-2-#27] INFO org.apache.ignite.internal.diagnostic - Exchange future waiting for coordinator response [crd=c68d7211-41ac-4364-81b5-46f55f62463e, topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
Remote node information:
General node info [id=c68d7211-41ac-4364-81b5-46f55f62463e, client=false, discoTopVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], time=06:10:57.094]
Partitions exchange info [readyVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
Last initialized exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 25374b3146cd/172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380655543, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1280119156], init=true, lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636067, centralizedAff=false, changeGlobalStateE=null, done=true, state=DONE, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=14, minorTopVer=0], hash=2059584827]]
Communication SPI statistics [rmtNode=2e01c750-0ae2-408b-8822-99aafc4ee434]
Communication SPI recovery descriptors:
[key=ConnectionKey [nodeId=2e01c750-0ae2-408b-8822-99aafc4ee434, idx=0, connCnt=0], msgsSent=2, msgsAckedByRmt=0, msgsRcvd=3, lastAcked=0, reserveCnt=1, descIdHash=1607104290]
Communication SPI clients:
[node=2e01c750-0ae2-408b-8822-99aafc4ee434, client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=50806, bytesSent=200521, bytesRcvd0=853, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=null, finished=false, hashCode=497542225, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#26]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=3, sentCnt=2, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=true, connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=3, sentCnt=2, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=true, connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false], super=GridNioSessionImpl [locAddr=/172.17.0.2:30000, rmtAddr=/10.10.10.146:46354, createTime=1510380637080, closeTime=0, bytesSent=55904, bytesRcvd=2170, bytesSent0=0, bytesRcvd0=853, sndSchedTime=1510380637080, lastSndTime=1510380647102, lastRcvTime=1510380657094, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@5a24c5cd, directMode=true], GridConnectionBytesVerifyFilter], accepted=true]], super=GridAbstractCommunicationClient [lastUsed=1510380637080, closed=false, connIdx=0]]]
NIO sessions statistics:
>> Selector info [idx=1, keysCnt=1, bytesRcvd=50806, bytesRcvd0=853, bytesSent=200521, bytesSent0=0]
Connection info [in=true, rmtAddr=/10.10.10.146:46354, locAddr=/172.17.0.2:30000, msgsSent=2, msgsAckedByRmt=0, descIdHash=1607104290, unackedMsgs=[GridDhtPartitionsFullMessage, IgniteDiagnosticMessage], msgsRcvd=3, lastAcked=0, descIdHash=1607104290, bytesRcvd=2170, bytesRcvd0=853, bytesSent=55904, bytesSent0=0, opQueueSize=0]
Exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 25374b3146cd/172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380655543, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1280119156], init=true, lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636067, centralizedAff=false, changeGlobalStateE=null, done=true, state=DONE, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=14, minorTopVer=0], hash=2059584827]]
Local communication statistics:
Communication SPI statistics [rmtNode=c68d7211-41ac-4364-81b5-46f55f62463e]
Communication SPI recovery descriptors:
[key=ConnectionKey [nodeId=c68d7211-41ac-4364-81b5-46f55f62463e, idx=0, connCnt=-1], msgsSent=3, msgsAckedByRmt=0, msgsRcvd=3, lastAcked=0, reserveCnt=7, descIdHash=980007565]
Communication SPI clients:
[node=c68d7211-41ac-4364-81b5-46f55f62463e, client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=2, bytesRcvd=64671, bytesSent=2146, bytesRcvd0=8795, bytesSent0=853, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=null, finished=false, hashCode=1240667813, interrupted=false, runner=grid-nio-worker-tcp-comm-2-#27]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=3, sentCnt=3, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=7, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=3, sentCnt=3, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=7, pairedConnections=false], super=GridNioSessionImpl [locAddr=/172.17.0.2:46354, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000, createTime=1510380637074, closeTime=0, bytesSent=2128, bytesRcvd=64671, bytesSent0=853, bytesRcvd0=8795, sndSchedTime=1510380637074, lastSndTime=1510380657088, lastRcvTime=1510380657088, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@204cfb1c, directMode=true], GridConnectionBytesVerifyFilter], accepted=false]], super=GridAbstractCommunicationClient [lastUsed=1510380637074, closed=false, connIdx=0]]]
NIO sessions statistics:
>> Selector info [idx=2, keysCnt=1, bytesRcvd=64671, bytesRcvd0=8795, bytesSent=2146, bytesSent0=853]
Connection info [in=false, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000, locAddr=/172.17.0.2:46354, msgsSent=3, msgsAckedByRmt=0, descIdHash=980007565, unackedMsgs=[GridDhtPartitionsSingleMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage], msgsRcvd=3, lastAcked=0, descIdHash=980007565, bytesRcvd=64671, bytesRcvd0=8795, bytesSent=2128, bytesSent0=853, opQueueSize=0]
2017-11-11 06:11:07:094 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:11:16:122 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380676090, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380676090, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]]
2017-11-11 06:11:17:095 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:11:17:095 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Ready affinity version: AffinityTopologyVersion [topVer=-1, minorTopVer=0]
2017-11-11 06:11:17:096 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Last exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380676090, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380676090, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]
2017-11-11 06:11:17:096 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - First 10 pending exchange futures [total=0]
2017-11-11 06:11:17:097 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Last 10 exchange futures (total: 1):
2017-11-11 06:11:17:097 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - >>> GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], evt=NODE_JOINED, evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380676090, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], done=false]
2017-11-11 06:11:17:097 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending transactions:
2017-11-11 06:11:17:097 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending explicit locks:
2017-11-11 06:11:17:097 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending cache futures:
2017-11-11 06:11:17:097 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending atomic cache futures:
2017-11-11 06:11:17:097 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending data streamer futures:
2017-11-11 06:11:17:097 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending transaction deadlock detection futures:
2017-11-11 06:11:17:101 +0000 [grid-nio-worker-tcp-comm-2-#27] INFO org.apache.ignite.internal.diagnostic - Exchange future waiting for coordinator response [crd=c68d7211-41ac-4364-81b5-46f55f62463e, topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
Remote node information:
General node info [id=c68d7211-41ac-4364-81b5-46f55f62463e, client=false, discoTopVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], time=06:11:17.097]
Partitions exchange info [readyVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
Last initialized exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 25374b3146cd/172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380675606, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1280119156], init=true, lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636067, centralizedAff=false, changeGlobalStateE=null, done=true, state=DONE, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=14, minorTopVer=0], hash=2059584827]]
Communication SPI statistics [rmtNode=2e01c750-0ae2-408b-8822-99aafc4ee434]
Communication SPI recovery descriptors:
[key=ConnectionKey [nodeId=2e01c750-0ae2-408b-8822-99aafc4ee434, idx=0, connCnt=0], msgsSent=3, msgsAckedByRmt=0, msgsRcvd=4, lastAcked=0, reserveCnt=1, descIdHash=1607104290]
Communication SPI clients:
[node=2e01c750-0ae2-408b-8822-99aafc4ee434, client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=51659, bytesSent=209316, bytesRcvd0=853, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=null, finished=false, hashCode=497542225, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#26]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=4, sentCnt=3, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=true, connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=4, sentCnt=3, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=true, connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false], super=GridNioSessionImpl [locAddr=/172.17.0.2:30000, rmtAddr=/10.10.10.146:46354, createTime=1510380637080, closeTime=0, bytesSent=64699, bytesRcvd=3023, bytesSent0=0, bytesRcvd0=853, sndSchedTime=1510380637080, lastSndTime=1510380657094, lastRcvTime=1510380677097, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@5a24c5cd, directMode=true], GridConnectionBytesVerifyFilter], accepted=true]], super=GridAbstractCommunicationClient [lastUsed=1510380637080, closed=false, connIdx=0]]]
NIO sessions statistics:
>> Selector info [idx=1, keysCnt=1, bytesRcvd=51659, bytesRcvd0=853, bytesSent=209316, bytesSent0=0]
Connection info [in=true, rmtAddr=/10.10.10.146:46354, locAddr=/172.17.0.2:30000, msgsSent=3, msgsAckedByRmt=0, descIdHash=1607104290, unackedMsgs=[GridDhtPartitionsFullMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage], msgsRcvd=4, lastAcked=0, descIdHash=1607104290, bytesRcvd=3023, bytesRcvd0=853, bytesSent=64699, bytesSent0=0, opQueueSize=0]
Exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 25374b3146cd/172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380675606, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1280119156], init=true, lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636067, centralizedAff=false, changeGlobalStateE=null, done=true, state=DONE, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=14, minorTopVer=0], hash=2059584827]]
Local communication statistics:
Communication SPI statistics [rmtNode=c68d7211-41ac-4364-81b5-46f55f62463e]
Communication SPI recovery descriptors:
[key=ConnectionKey [nodeId=c68d7211-41ac-4364-81b5-46f55f62463e, idx=0, connCnt=-1], msgsSent=4, msgsAckedByRmt=0, msgsRcvd=4, lastAcked=0, reserveCnt=7, descIdHash=980007565]
Communication SPI clients:
[node=c68d7211-41ac-4364-81b5-46f55f62463e, client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=2, bytesRcvd=73491, bytesSent=2999, bytesRcvd0=8820, bytesSent0=853, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=null, finished=false, hashCode=1240667813, interrupted=false, runner=grid-nio-worker-tcp-comm-2-#27]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=4, sentCnt=4, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=7, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=4, sentCnt=4, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=7, pairedConnections=false], super=GridNioSessionImpl [locAddr=/172.17.0.2:46354, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000, createTime=1510380637074, closeTime=0, bytesSent=2981, bytesRcvd=73491, bytesSent0=853, bytesRcvd0=8820, sndSchedTime=1510380637074, lastSndTime=1510380677093, lastRcvTime=1510380677093, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@204cfb1c, directMode=true], GridConnectionBytesVerifyFilter], accepted=false]], super=GridAbstractCommunicationClient [lastUsed=1510380637074, closed=false, connIdx=0]]]
NIO sessions statistics:
>> Selector info [idx=2, keysCnt=1, bytesRcvd=73491, bytesRcvd0=8820, bytesSent=2999, bytesSent0=853]
Connection info [in=false, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000, locAddr=/172.17.0.2:46354, msgsSent=4, msgsAckedByRmt=0, descIdHash=980007565, unackedMsgs=[GridDhtPartitionsSingleMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage], msgsRcvd=4, lastAcked=0, descIdHash=980007565, bytesRcvd=73491, bytesRcvd0=8820, bytesSent=2981, bytesSent0=853, opQueueSize=0]
2017-11-11 06:11:27:098 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:11:36:123 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380696100, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380696100, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]]
2017-11-11 06:11:37:099 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:11:47:099 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:11:56:124 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380716114, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380716114, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]]
2017-11-11 06:11:57:099 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:11:57:099 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Ready affinity version: AffinityTopologyVersion [topVer=-1, minorTopVer=0]
2017-11-11 06:11:57:100 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Last exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380716114, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380716114, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]
2017-11-11 06:11:57:100 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - First 10 pending exchange futures [total=0]
2017-11-11 06:11:57:100 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Last 10 exchange futures (total: 1):
2017-11-11 06:11:57:100 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - >>> GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], evt=NODE_JOINED, evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380716114, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], done=false]
2017-11-11 06:11:57:100 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending transactions:
2017-11-11 06:11:57:100 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending explicit locks:
2017-11-11 06:11:57:100 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending cache futures:
2017-11-11 06:11:57:101 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending atomic cache futures:
2017-11-11 06:11:57:101 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending data streamer futures:
2017-11-11 06:11:57:101 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending transaction deadlock detection futures:
2017-11-11 06:11:57:104 +0000 [grid-nio-worker-tcp-comm-2-#27] INFO org.apache.ignite.internal.diagnostic - Exchange future waiting for coordinator response [crd=c68d7211-41ac-4364-81b5-46f55f62463e, topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
Remote node information:
General node info [id=c68d7211-41ac-4364-81b5-46f55f62463e, client=false, discoTopVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], time=06:11:57.106]
Partitions exchange info [readyVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
Last initialized exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 25374b3146cd/172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380715748, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1280119156], init=true, lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636067, centralizedAff=false, changeGlobalStateE=null, done=true, state=DONE, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=14, minorTopVer=0], hash=2059584827]]
Communication SPI statistics [rmtNode=2e01c750-0ae2-408b-8822-99aafc4ee434]
Communication SPI recovery descriptors:
[key=ConnectionKey [nodeId=2e01c750-0ae2-408b-8822-99aafc4ee434, idx=0, connCnt=0], msgsSent=4, msgsAckedByRmt=0, msgsRcvd=5, lastAcked=0, reserveCnt=1, descIdHash=1607104290]
Communication SPI clients:
[node=2e01c750-0ae2-408b-8822-99aafc4ee434, client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=52512, bytesSent=218136, bytesRcvd0=853, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=null, finished=false, hashCode=497542225, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#26]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=5, sentCnt=4, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=true, connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=5, sentCnt=4, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=true, connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false], super=GridNioSessionImpl [locAddr=/172.17.0.2:30000, rmtAddr=/10.10.10.146:46354, createTime=1510380637080, closeTime=0, bytesSent=73519, bytesRcvd=3876, bytesSent0=0, bytesRcvd0=853, sndSchedTime=1510380637080, lastSndTime=1510380677097, lastRcvTime=1510380717106, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@5a24c5cd, directMode=true], GridConnectionBytesVerifyFilter], accepted=true]], super=GridAbstractCommunicationClient [lastUsed=1510380637080, closed=false, connIdx=0]]]
NIO sessions statistics:
>> Selector info [idx=1, keysCnt=1, bytesRcvd=52512, bytesRcvd0=853, bytesSent=218136, bytesSent0=0]
Connection info [in=true, rmtAddr=/10.10.10.146:46354, locAddr=/172.17.0.2:30000, msgsSent=4, msgsAckedByRmt=0, descIdHash=1607104290, unackedMsgs=[GridDhtPartitionsFullMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage], msgsRcvd=5, lastAcked=0, descIdHash=1607104290, bytesRcvd=3876, bytesRcvd0=853, bytesSent=73519, bytesSent0=0, opQueueSize=0]
Exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 25374b3146cd/172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380715748, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1280119156], init=true, lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636067, centralizedAff=false, changeGlobalStateE=null, done=true, state=DONE, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=14, minorTopVer=0], hash=2059584827]]
Local communication statistics:
Communication SPI statistics [rmtNode=c68d7211-41ac-4364-81b5-46f55f62463e]
Communication SPI recovery descriptors:
[key=ConnectionKey [nodeId=c68d7211-41ac-4364-81b5-46f55f62463e, idx=0, connCnt=-1], msgsSent=5, msgsAckedByRmt=0, msgsRcvd=5, lastAcked=0, reserveCnt=7, descIdHash=980007565]
Communication SPI clients:
[node=c68d7211-41ac-4364-81b5-46f55f62463e, client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=2, bytesRcvd=82336, bytesSent=3852, bytesRcvd0=8845, bytesSent0=853, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=null, finished=false, hashCode=1240667813, interrupted=false, runner=grid-nio-worker-tcp-comm-2-#27]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=5, sentCnt=5, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=7, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=5, sentCnt=5, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=7, pairedConnections=false], super=GridNioSessionImpl [locAddr=/172.17.0.2:46354, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000, createTime=1510380637074, closeTime=0, bytesSent=3834, bytesRcvd=82336, bytesSent0=853, bytesRcvd0=8845, sndSchedTime=1510380637074, lastSndTime=1510380717097, lastRcvTime=1510380717097, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@204cfb1c, directMode=true], GridConnectionBytesVerifyFilter], accepted=false]], super=GridAbstractCommunicationClient [lastUsed=1510380637074, closed=false, connIdx=0]]]
NIO sessions statistics:
>> Selector info [idx=2, keysCnt=1, bytesRcvd=82336, bytesRcvd0=8845, bytesSent=3852, bytesSent0=853]
Connection info [in=false, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000, locAddr=/172.17.0.2:46354, msgsSent=5, msgsAckedByRmt=0, descIdHash=980007565, unackedMsgs=[GridDhtPartitionsSingleMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage], msgsRcvd=5, lastAcked=0, descIdHash=980007565, bytesRcvd=82336, bytesRcvd0=8845, bytesSent=3834, bytesSent0=853, opQueueSize=0]
2017-11-11 06:12:07:101 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:12:16:124 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380735812, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380735812, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]]
2017-11-11 06:12:17:102 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:12:27:102 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:12:36:125 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380755867, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380755867, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]]
2017-11-11 06:12:37:103 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:12:47:103 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:12:56:126 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380775932, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380775932, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]]
2017-11-11 06:12:57:103 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:13:07:104 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:13:16:127 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380796000, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380796000, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]]
2017-11-11 06:13:17:104 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:13:27:105 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:13:27:105 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Ready affinity version: AffinityTopologyVersion [topVer=-1, minorTopVer=0]
2017-11-11 06:13:27:105 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Last exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380806176, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380806176, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]
2017-11-11 06:13:27:105 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - First 10 pending exchange futures [total=0]
2017-11-11 06:13:27:105 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Last 10 exchange futures (total: 1):
2017-11-11 06:13:27:105 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - >>> GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], evt=NODE_JOINED, evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380806176, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], done=false]
2017-11-11 06:13:27:106 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending transactions:
2017-11-11 06:13:27:106 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending explicit locks:
2017-11-11 06:13:27:106 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending cache futures:
2017-11-11 06:13:27:106 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending atomic cache futures:
2017-11-11 06:13:27:106 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending data streamer futures:
2017-11-11 06:13:27:106 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Pending transaction deadlock detection futures:
2017-11-11 06:13:27:110 +0000 [grid-nio-worker-tcp-comm-2-#27] INFO org.apache.ignite.internal.diagnostic - Exchange future waiting for coordinator response [crd=c68d7211-41ac-4364-81b5-46f55f62463e, topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
Remote node information:
General node info [id=c68d7211-41ac-4364-81b5-46f55f62463e, client=false, discoTopVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], time=06:13:27.108]
Partitions exchange info [readyVer=AffinityTopologyVersion [topVer=14, minorTopVer=0]]
Last initialized exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 25374b3146cd/172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380806033, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1280119156], init=true, lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636067, centralizedAff=false, changeGlobalStateE=null, done=true, state=DONE, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=14, minorTopVer=0], hash=2059584827]]
Communication SPI statistics [rmtNode=2e01c750-0ae2-408b-8822-99aafc4ee434]
Communication SPI recovery descriptors:
[key=ConnectionKey [nodeId=2e01c750-0ae2-408b-8822-99aafc4ee434, idx=0, connCnt=0], msgsSent=5, msgsAckedByRmt=0, msgsRcvd=6, lastAcked=5, reserveCnt=1, descIdHash=1607104290]
Communication SPI clients:
[node=2e01c750-0ae2-408b-8822-99aafc4ee434, client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=1, bytesRcvd=53365, bytesSent=226991, bytesRcvd0=853, bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=null, finished=false, hashCode=497542225, interrupted=false, runner=grid-nio-worker-tcp-comm-1-#26]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=6, sentCnt=5, reserved=true, lastAck=5, nodeLeft=false, node=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=true, connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=6, sentCnt=5, reserved=true, lastAck=5, nodeLeft=false, node=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=true, connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false], super=GridNioSessionImpl [locAddr=/172.17.0.2:30000, rmtAddr=/10.10.10.146:46354, createTime=1510380637080, closeTime=0, bytesSent=82374, bytesRcvd=4729, bytesSent0=0, bytesRcvd0=853, sndSchedTime=1510380637080, lastSndTime=1510380738072, lastRcvTime=1510380807108, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@5a24c5cd, directMode=true], GridConnectionBytesVerifyFilter], accepted=true]], super=GridAbstractCommunicationClient [lastUsed=1510380637080, closed=false, connIdx=0]]]
NIO sessions statistics:
>> Selector info [idx=1, keysCnt=1, bytesRcvd=53365, bytesRcvd0=853, bytesSent=226991, bytesSent0=0]
Connection info [in=true, rmtAddr=/10.10.10.146:46354, locAddr=/172.17.0.2:30000, msgsSent=5, msgsAckedByRmt=0, descIdHash=1607104290, unackedMsgs=[GridDhtPartitionsFullMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage, IgniteDiagnosticMessage], msgsRcvd=6, lastAcked=5, descIdHash=1607104290, bytesRcvd=4729, bytesRcvd0=853, bytesSent=82374, bytesSent0=0, opQueueSize=0]
Exchange future: GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 25374b3146cd/172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380806033, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=c68d7211, msg=Node joined: TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380635723, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], type=NODE_JOINED, tstamp=1510380636067], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1280119156], init=true, lastVer=GridCacheVersion [topVer=0, order=1510380635210, nodeOrder=0], partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636067, centralizedAff=false, changeGlobalStateE=null, done=true, state=DONE, evtLatch=0, remaining=[], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=AffinityTopologyVersion [topVer=14, minorTopVer=0], hash=2059584827]]
Local communication statistics:
Communication SPI statistics [rmtNode=c68d7211-41ac-4364-81b5-46f55f62463e]
Communication SPI recovery descriptors:
[key=ConnectionKey [nodeId=c68d7211-41ac-4364-81b5-46f55f62463e, idx=0, connCnt=-1], msgsSent=6, msgsAckedByRmt=5, msgsRcvd=6, lastAcked=0, reserveCnt=7, descIdHash=980007565]
Communication SPI clients:
[node=c68d7211-41ac-4364-81b5-46f55f62463e, client=GridTcpNioCommunicationClient [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker [super=AbstractNioClientWorker [idx=2, bytesRcvd=91216, bytesSent=4705, bytesRcvd0=8870, bytesSent0=853, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=null, finished=false, hashCode=1240667813, interrupted=false, runner=grid-nio-worker-tcp-comm-2-#27]]], writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=GridNioRecoveryDescriptor [acked=5, resendCnt=0, rcvCnt=6, sentCnt=6, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=7, pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=5, resendCnt=0, rcvCnt=6, sentCnt=6, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], connected=false, connectCnt=1, queueLimit=4096, reserveCnt=7, pairedConnections=false], super=GridNioSessionImpl [locAddr=/172.17.0.2:46354, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000, createTime=1510380637074, closeTime=0, bytesSent=4687, bytesRcvd=91216, bytesSent0=853, bytesRcvd0=8870, sndSchedTime=1510380637074, lastSndTime=1510380807098, lastRcvTime=1510380807108, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=org.apache.ignite.internal.util.nio.GridDirectParser@204cfb1c, directMode=true], GridConnectionBytesVerifyFilter], accepted=false]], super=GridAbstractCommunicationClient [lastUsed=1510380637074, closed=false, connIdx=0]]]
NIO sessions statistics:
>> Selector info [idx=2, keysCnt=1, bytesRcvd=91216, bytesRcvd0=8870, bytesSent=4705, bytesSent0=853]
Connection info [in=false, rmtAddr=ip-10-10-100-251.eu-central-1.compute.internal/10.10.100.251:30000, locAddr=/172.17.0.2:46354, msgsSent=6, msgsAckedByRmt=5, descIdHash=980007565, unackedMsgs=[IgniteDiagnosticMessage], msgsRcvd=6, lastAcked=0, descIdHash=980007565, bytesRcvd=91216, bytesRcvd0=8870, bytesSent=4687, bytesSent0=853, opQueueSize=0]
2017-11-11 06:13:36:128 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380816057, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380816057, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]]
2017-11-11 06:13:37:106 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:13:47:107 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:13:56:128 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380834118, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380834118, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]]
2017-11-11 06:13:57:107 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:14:07:107 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:14:16:129 +0000 [main] WARN org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager - Still waiting for initial partition map exchange [fut=GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380854188, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], crd=TcpDiscoveryNode [id=c68d7211-41ac-4364-81b5-46f55f62463e, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=1, intOrder=1, lastExchangeTime=1510380636000, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=2e01c750-0ae2-408b-8822-99aafc4ee434, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, 8a48b3b99a6e/172.17.0.2:31000], discPort=31000, order=14, intOrder=10, lastExchangeTime=1510380854188, loc=true, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false], topVer=14, nodeId8=2e01c750, msg=null, type=NODE_JOINED, tstamp=1510380636101], nodeId=2e01c750, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=true, hash=1989363751], init=true, lastVer=null, partReleaseFut=PartitionReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[ExplicitLockReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], TxReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], AtomicUpdateReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]], DataStreamerReleaseFuture [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], futures=[]]]], exchActions=null, affChangeMsg=null, initTs=1510380636112, centralizedAff=false, changeGlobalStateE=null, done=false, state=DONE, evtLatch=0, remaining=[f939cfa6-2389-41d4-bc3e-64baa1e98d0f, 41dcf661-735f-49b6-8e93-bf56901f1b4d, fbc7ce21-e321-45a5-9c2c-9c9ced4aaf2d, c68d7211-41ac-4364-81b5-46f55f62463e, 9593b93f-c51e-4b52-b245-9ac139297465], super=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=1825950807]]]
2017-11-11 06:14:17:108 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:14:27:108 +0000 [exchange-worker-#42] WARN org.apache.ignite.internal.diagnostic - Failed to wait for partition map exchange [topVer=AffinityTopologyVersion [topVer=14, minorTopVer=0], node=2e01c750-0ae2-408b-8822-99aafc4ee434]. Dumping pending objects that might be the cause:
2017-11-11 06:14:27:529 +0000 [tcp-disco-srvr-#3] INFO org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - TCP discovery accepted incoming connection [rmtAddr=/10.10.55.111, rmtPort=41183]
2017-11-11 06:14:27:529 +0000 [tcp-disco-srvr-#3] INFO org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - TCP discovery spawning a new thread for connection [rmtAddr=/10.10.55.111, rmtPort=41183]
2017-11-11 06:14:27:529 +0000 [tcp-disco-sock-reader-#6] INFO org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Started serving remote node connection [rmtAddr=/10.10.55.111:41183, rmtPort=41183]
2017-11-11 06:14:27:552 +0000 [tcp-disco-sock-reader-#6] INFO org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi - Finished serving remote node connection [rmtAddr=/10.10.55.111:41183, rmtPort=41183
2017-11-11 06:14:27:891 +0000 [disco-event-worker-#41] INFO org.apache.ignite.internal.managers.discovery.GridDiscoveryManager - Added new node to topology: TcpDiscoveryNode [id=f89dfc2f-14f1-4949-9e3d-ef6f4169f888, addrs=[127.0.0.1, 172.17.0.2], sockAddrs=[/127.0.0.1:31000, /172.17.0.2:31000], discPort=31000, order=15, intOrder=11, lastExchangeTime=1510380867617, loc=false, ver=2.3.0#20171028-sha1:8add7fd5, isClient=false]
2017-11-11 06:14:27:891 +0000 [disco-event-worker-#41] INFO org.apache.ignite.internal.managers.discovery.GridDiscoveryManager - Topology snapshot [ver=15, servers=7, clients=0, CPUs=8, heap=28.0GB]

Thankx
Lukas


----

Lukas Lentner, B. Sc.
St.-Cajetan-Straße 13
81669 München
Deutschland
Fon:     +49 / 89  / 44 38 61 27
Mobile:  +49 / 176 / 24 77 09 22
E-Mail:  Kontakt@LukasLentner.de
Website: www.LukasLentner.de

IBAN:    DE33 7019 0000 0001 1810 17
BIC:     GENODEF1M01 (Münchner Bank)


Mime
View raw message