Return-Path: X-Original-To: apmail-ignite-issues-archive@minotaur.apache.org Delivered-To: apmail-ignite-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 99AE918BCD for ; Mon, 16 Nov 2015 13:47:11 +0000 (UTC) Received: (qmail 22931 invoked by uid 500); 16 Nov 2015 13:47:11 -0000 Delivered-To: apmail-ignite-issues-archive@ignite.apache.org Received: (qmail 22885 invoked by uid 500); 16 Nov 2015 13:47:11 -0000 Mailing-List: contact issues-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list issues@ignite.apache.org Received: (qmail 22874 invoked by uid 99); 16 Nov 2015 13:47:11 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 Nov 2015 13:47:11 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 0F54B2C0450 for ; Mon, 16 Nov 2015 13:47:11 +0000 (UTC) Date: Mon, 16 Nov 2015 13:47:11 +0000 (UTC) From: "Pavel Tupitsyn (JIRA)" To: issues@ignite.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (IGNITE-1924) Incomplete marshaller cache rebalancing causes Grid hangs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/IGNITE-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-1924: ------------------------------------ Summary: Incomplete marshaller cache rebalancing causes Grid hangs (was: Uncompleme marshaller cashe rebalancing cause Grid hangs) > Incomplete marshaller cache rebalancing causes Grid hangs > --------------------------------------------------------- > > Key: IGNITE-1924 > URL: https://issues.apache.org/jira/browse/IGNITE-1924 > Project: Ignite > Issue Type: Bug > Reporter: Anton Vinogradov > Assignee: Anton Vinogradov > Priority: Critical > > End of the log. > [11:49:32] : [org.apache.ignite:ignite-core] [11:49:32,947][INFO ][exchange-worker-#220584%tcp.IgniteCacheSslStartStopSelfTest3%][GridDhtPartitionDemander] Starting rebalancing [cache=ignite-marshaller-sys-cache, mode=SYNC, fromNode=108bffdb-1c1e-49aa-9525-b434784fa001, partitionsCount=7, topology=AffinityTopologyVersion [topVer=594, minorTopVer=0], updateSeq=1] > [11:49:32] : [org.apache.ignite:ignite-core] [11:49:32,962][INFO ][exchange-worker-#220584%tcp.IgniteCacheSslStartStopSelfTest3%][GridDhtPartitionDemander] Starting rebalancing [cache=ignite-marshaller-sys-cache, mode=SYNC, fromNode=20660c29-91a1-4279-9dc1-88d192bc6002, partitionsCount=6, topology=AffinityTopologyVersion [topVer=594, minorTopVer=0], updateSeq=1] > [11:49:32] : [org.apache.ignite:ignite-core] [11:49:32,962][INFO ][exchange-worker-#220584%tcp.IgniteCacheSslStartStopSelfTest3%][GridDhtPartitionDemander] Starting rebalancing [cache=ignite-marshaller-sys-cache, mode=SYNC, fromNode=00b3a75a-074d-46a5-a158-3956c0ec4000, partitionsCount=7, topology=AffinityTopologyVersion [topVer=594, minorTopVer=0], updateSeq=1] > [11:49:32] : [org.apache.ignite:ignite-core] [11:49:32,963][INFO ][ignite-#220587%marshaller-cache-tcp.IgniteCacheSslStartStopSelfTest3%][GridDhtPartitionDemander] Completed rebalancing [cache=ignite-marshaller-sys-cache, fromNode=00b3a75a-074d-46a5-a158-3956c0ec4000, topology=AffinityTopologyVersion [topVer=594, minorTopVer=0], time=21 ms] > [11:49:32] : [org.apache.ignite:ignite-core] [11:49:32,963][INFO ][ignite-#220586%marshaller-cache-tcp.IgniteCacheSslStartStopSelfTest3%][GridDhtPartitionDemander] Completed rebalancing [cache=ignite-marshaller-sys-cache, fromNode=108bffdb-1c1e-49aa-9525-b434784fa001, topology=AffinityTopologyVersion [topVer=594, minorTopVer=0], time=21 ms] > Hang on: > [11:51:56] : [org.apache.ignite:ignite-core] Thread [name="ignite-#220562%sys-tcp.IgniteCacheSslStartStopSelfTest3%", id=287517, state=WAITING, blockCnt=0, waitCnt=3] > [11:51:56] : [org.apache.ignite:ignite-core] Lock [object=o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander$RebalanceFuture@b402f89, ownerName=null, ownerId=-1] > [11:51:56] : [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native Method) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:157) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:115) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.waitForCacheRebalancing(GridDhtPartitionDemander.java:265) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.access$400(GridDhtPartitionDemander.java:85) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander$3.call(GridDhtPartitionDemander.java:323) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander$3.call(GridDhtPartitionDemander.java:320) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker$1.call(GridCachePartitionExchangeManager.java:1386) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker$1.call(GridCachePartitionExchangeManager.java:1377) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6371) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:929) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:110) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [11:51:56] : [org.apache.ignite:ignite-core] at java.lang.Thread.run(Thread.java:745) > [11:51:56] : [org.apache.ignite:ignite-core] > [11:51:56] : [org.apache.ignite:ignite-core] Locked synchronizers: > [11:51:56] : [org.apache.ignite:ignite-core] java.util.concurrent.ThreadPoolExecutor$Worker@7a9245cc > [11:51:56] : [org.apache.ignite:ignite-core] Thread [name="ignite-#220561%sys-tcp.IgniteCacheSslStartStopSelfTest3%", id=287516, state=WAITING, blockCnt=0, waitCnt=2] > [11:51:56] : [org.apache.ignite:ignite-core] Lock [object=java.util.concurrent.CountDownLatch$Sync@22f0d124, ownerName=null, ownerId=-1] > [11:51:56] : [org.apache.ignite:ignite-core] at sun.misc.Unsafe.park(Native Method) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.util.IgniteUtils.awaitQuiet(IgniteUtils.java:7201) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.MarshallerContextImpl.className(MarshallerContextImpl.java:143) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.MarshallerContextAdapter.getClass(MarshallerContextAdapter.java:174) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.marshaller.optimized.OptimizedMarshallerUtils.classDescriptor(OptimizedMarshallerUtils.java:257) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:309) > [11:51:56] : [org.apache.ignite:ignite-core] at java.io.ObjectInputStream.readObject(ObjectInputStream.java:364) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.marshaller.optimized.OptimizedMarshaller.unmarshal(OptimizedMarshaller.java:248) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.GridCacheMessage.unmarshalCollection(GridCacheMessage.java:599) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateRequest.finishUnmarshal(GridNearAtomicUpdateRequest.java:584) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:996) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:268) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:197) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:76) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.cache.GridCacheIoManager$1$1$1.run(GridCacheIoManager.java:150) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6427) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:788) > [11:51:56] : [org.apache.ignite:ignite-core] at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:110) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [11:51:56] : [org.apache.ignite:ignite-core] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [11:51:56] : [org.apache.ignite:ignite-core] at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)