From issues-return-98449-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Fri Sep 6 06:23:03 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1B503180674 for ; Fri, 6 Sep 2019 08:23:03 +0200 (CEST) Received: (qmail 85226 invoked by uid 500); 6 Sep 2019 16:47:52 -0000 Mailing-List: contact issues-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list issues@ignite.apache.org Received: (qmail 85202 invoked by uid 99); 6 Sep 2019 16:47:52 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Sep 2019 16:47:52 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 9AC4BE312F for ; Fri, 6 Sep 2019 06:23:01 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 56585780907 for ; Fri, 6 Sep 2019 06:23:00 +0000 (UTC) Date: Fri, 6 Sep 2019 06:23:00 +0000 (UTC) From: "Aditya Gupta (Jira)" To: issues@ignite.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (IGNITE-12143) Vacuum error. class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Aditya Gupta created IGNITE-12143: ------------------------------------- Summary: Vacuum error. class org.apache.ignite.internal.trans= actions.IgniteTxMvccVersionCheckedException Key: IGNITE-12143 URL: https://issues.apache.org/jira/browse/IGNITE-12143 Project: Ignite Issue Type: Bug Components: mvcc Affects Versions: 2.7.5 Reporter: Aditya Gupta Fix For: 2.7.5 2019-09-05 17:54:46.576 INFO STDERR [Thread-18] : [17:= 54:46] (err) Failed to execute compound future reducer: GridCompoundFuture = [rdc=3Dnull, initFlag=3D0, lsnrCalls=3D0, done=3Dfalse, cancelled=3Dfalse, = err=3Dnull, futs=3D[false, false, false, false, false, false, false, false,= false, false, false, false, false, false, false, false, false, false, fals= e, false, true]]class org.apache.ignite.IgniteCheckedException: class org.a= pache.ignite.transactions.TransactionRollbackException: Transaction has bee= n rolled back: 108bb6a2d61-00000000-0aad-7c5d-0000-000000000001 2019-09-05 17:54:46.576 INFO STDERR [Thread-18] : =C2= =A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.util.IgniteUtils.cast(I= gniteUtils.java:7429) 2019-09-05 17:54:46.576 INFO STDERR [Thread-18] : =C2= =A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.closure.Grid= ClosureProcessor$2.body(GridClosureProcessor.java:975) 2019-09-05 17:54:46.576 INFO STDERR [Thread-18] : =C2= =A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.util.worker.GridWorker.= run(GridWorker.java:120) 2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : =C2= =A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.util.StripedExecutor$St= ripe.body(StripedExecutor.java:505) 2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : =C2= =A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.util.worker.GridWorker.= run(GridWorker.java:120) 2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : =C2= =A0=C2=A0=C2=A0=C2=A0 at java.lang.Thread.run(Thread.java:745) 2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : Caused by: ja= vax.cache.CacheException: class org.apache.ignite.transactions.TransactionR= ollbackException: Transaction has been rolled back: 108bb6a2d61-00000000-0a= ad-7c5d-0000-000000000001 2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Gri= dCacheUtils.convertToCacheException(GridCacheUtils.java:1337) 2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Ign= iteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1758) 2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Ign= iteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1108) 2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Gat= ewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:820) 2019-09-05 17:54:46.577 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.datastrea= mer.DataStreamerCacheUpdaters$Individual.receive(DataStreamerCacheUpdaters.= java:121) 2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.datastrea= mer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140) 2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.util.IgniteUtils.wra= pThreadLoader(IgniteUtils.java:6817) 2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.closure.G= ridClosureProcessor$2.body(GridClosureProcessor.java:967) 2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 ... 4 more 2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : Caused by: cl= ass org.apache.ignite.transactions.TransactionRollbackException: Transactio= n has been rolled back: 108bb6a2d61-00000000-0aad-7c5d-0000-000000000001 2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.util.IgniteUtils$11.= apply(IgniteUtils.java:920) 2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.util.IgniteUtils$11.= apply(IgniteUtils.java:918) 2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 ... 12 more 2019-09-05 17:54:46.578 INFO STDERR [Thread-18] : Caused by: cl= ass org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedExceptio= n: Transaction has been rolled back: 108bb6a2d61-00000000-0aad-7c5d-0000-00= 0000000001 2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Gri= dCacheAdapter.syncOp(GridCacheAdapter.java:4248) 2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Gri= dCacheAdapter.put0(GridCacheAdapter.java:2468) 2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Gri= dCacheAdapter.put(GridCacheAdapter.java:2449) 2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Gri= dCacheAdapter.put(GridCacheAdapter.java:2426) 2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Ign= iteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1105) 2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 ... 9 more 2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : Caused by: cl= ass org.apache.ignite.IgniteCheckedException: Runtime failure on bounds: [l= ower=3DMvccMaxSearchRow [], upper=3DMvccMinSearchRow []] 2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree.visit(BPlusTree.java:1061) 2019-09-05 17:54:46.579 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Ign= iteCacheOffheapManagerImpl$CacheDataStoreImpl.mvccUpdate(IgniteCacheOffheap= ManagerImpl.java:1939) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Ign= iteCacheOffheapManagerImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:53= 2) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Gri= dCacheMapEntry.mvccSet(GridCacheMapEntry.java:1104) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.dht.GridDhtTxAbstractEnlistFuture.continueLoop(GridDhtTxAbstractEn= listFuture.java:456) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.dht.GridDhtTxAbstractEnlistFuture.init(GridDhtTxAbstractEnlistFutu= re.java:364) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.near.GridNearTxEnlistFuture.enlistLocal(GridNearTxEnlistFuture.jav= a:518) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.near.GridNearTxEnlistFuture.sendBatch(GridNearTxEnlistFuture.java:= 413) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.near.GridNearTxEnlistFuture.sendNextBatches(GridNearTxEnlistFuture= .java:168) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.near.GridNearTxEnlistFuture.map(GridNearTxEnlistFuture.java:144) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.near.GridNearTxAbstractEnlistFuture.mapOnTopology(GridNearTxAbstra= ctEnlistFuture.java:331) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.near.GridNearTxAbstractEnlistFuture.init(GridNearTxAbstractEnlistF= uture.java:246) 2019-09-05 17:54:46.580 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.near.GridNearTxLocal.updateAsync(GridNearTxLocal.java:2099) 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.near.GridNearTxLocal.mvccPutAllAsync0(GridNearTxLocal.java:802) 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.near.GridNearTxLocal.putAsync0(GridNearTxLocal.java:582) 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.dis= tributed.near.GridNearTxLocal.putAsync(GridNearTxLocal.java:448) 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Gri= dCacheAdapter$22.op(GridCacheAdapter.java:2470) 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Gri= dCacheAdapter$22.op(GridCacheAdapter.java:2468) 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.Gri= dCacheAdapter.syncOp(GridCacheAdapter.java:4233) 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 ... 13 more 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : Caused by: cl= ass org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedExcep= tion: Unexpected state: [state=3D0, rowVer=3D1567661724689:237333229:1, txV= er=3D1567661724689:237689217:1, localNodeId=3Deb509bae-864c-46bc-a85e-57928= b639f67] 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.mvc= c.MvccUtils.unexpectedStateException(MvccUtils.java:317) 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.mvc= c.MvccUtils.unexpectedStateException(MvccUtils.java:304) 2019-09-05 17:54:46.581 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.tre= e.mvcc.data.MvccUpdateDataRow.visit(MvccUpdateDataRow.java:413) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3021) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree$TreeVisitor.init(BPlusTree.java:2990) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree$TreeVisitor.notFound(BPlusTree.java:2946) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree$Search.run0(BPlusTree.java:306) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5615) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree$Search.run(BPlusTree.java:271) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5600) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.util.PageHandler.readPage(PageHandler.java:159) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.DataStructure.read(DataStructure.java:334) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree.visitDown(BPlusTree.java:2585) 2019-09-05 17:54:46.582 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree.visitDown(BPlusTree.java:2594) 2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree.doVisit(BPlusTree.java:2552) 2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree.access$8200(BPlusTree.java:90) 2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3148) 2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree$TreeVisitor.access$5100(BPlusTree.java:2908) 2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.processors.cache.per= sistence.tree.BPlusTree.visit(BPlusTree.java:1058) 2019-09-05 17:54:46.583 INFO STDERR [Thread-18] : =C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 ... 31 more 2019-09-05 17:54:46.692 ERROR org.apache.ignite.internal.processors.cache.m= vcc.MvccProcessorImpl [vacuum-scheduler-#45%rates-marple-pnlswaps%] : Vacuu= m error. class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedExc= eption: Unexpected state: [state=3D0, rowVer=3D1567661724689:237333229:1, l= ocalNodeId=3Deb509bae-864c-46bc-a85e-57928b639f67] =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:317) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:288) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.r= un(IgniteCacheOffheapManagerImpl.java:3312) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.r= un(IgniteCacheOffheapManagerImpl.java:3286) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java= :279) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.persistence.DataStructure.write(DataStructure.java:256) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.persistence.freelist.AbstractFreeList.updateDataRow(Abstract= FreeList.java:555) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.persistence.RowStore.updateDataRow(RowStore.java:138) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxSta= te(IgniteCacheOffheapManagerImpl.java:2469) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImp= l.java:2425) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccPro= cessorImpl.java:2285) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.j= ava:2208) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.ut= il.worker.GridWorker.run(GridWorker.java:120) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.lang.Thread.run(Thread.j= ava:745) 2019-09-05 17:54:47.652 ERROR root [data-streamer-stripe-2-#11%rates-marple= -pnlswaps%] : Critical system error detected. Will be handled accordingly t= o configured handler [hnd=3DStopNodeOrHaltFailureHandler [tryStop=3Dfalse, = timeout=3D0, super=3DAbstractFailureHandler [ignoredFailureTypes=3D[SYSTEM_= WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=3DFailureC= ontext [type=3DCRITICAL_ERROR, err=3Dclass o.a.i.IgniteCheckedException: Ma= ximum number of retries 1000 reached for Remove operation (the tree may be = corrupted). Increase IGNITE_BPLUS_TREE_LOCK_RETRIES system property if you = regularly see this message (current value is 1000).]] class org.apache.ignite.IgniteCheckedException: Maximum number of retries 1= 000 reached for Remove operation (the tree may be corrupted). Increase IGNI= TE_BPLUS_TREE_LOCK_RETRIES system property if you regularly see this messag= e (current value is 1000). =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.persistence.tree.BPlusTree$Get.checkLockRetry(BPlusTree.java= :2813) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.persistence.tree.BPlusTree.removeDown(BPlusTree.java:1997) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.persistence.tree.BPlusTree.removeDown(BPlusTree.java:2023) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.persistence.tree.BPlusTree.removeDown(BPlusTree.java:2023) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.persistence.tree.BPlusTree.doRemove(BPlusTree.java:1934) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.persistence.tree.BPlusTree.removex(BPlusTree.java:1764) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.query.h2.database.H2TreeIndex.removex(H2TreeIndex.java:345) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.query.h2.opt.GridH2Table.remove(GridH2Table.java:521) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.query.h2.IgniteH2Indexing.remove(IgniteH2Indexing.java:797) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.query.GridQueryProcessor.remove(GridQueryProcessor.java:2586) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.query.GridCacheQueryManager.remove(GridCacheQueryManager.jav= a:435) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.cleanup(Ign= iteCacheOffheapManagerImpl.java:2445) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.mvccUpdate(= IgniteCacheOffheapManagerImpl.java:2034) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.IgniteCacheOffheapManagerImpl.mvccUpdate(IgniteCacheOffheapM= anagerImpl.java:532) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.GridCacheMapEntry.mvccSet(GridCacheMapEntry.java:1104) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.continueLoop(G= ridDhtTxAbstractEnlistFuture.java:456) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.init(GridDhtTx= AbstractEnlistFuture.java:364) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.near.GridNearTxEnlistFuture.enlistLocal(GridNear= TxEnlistFuture.java:518) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.near.GridNearTxEnlistFuture.sendBatch(GridNearTx= EnlistFuture.java:413) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.near.GridNearTxEnlistFuture.sendNextBatches(Grid= NearTxEnlistFuture.java:168) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.near.GridNearTxEnlistFuture.map(GridNearTxEnlist= Future.java:144) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.near.GridNearTxAbstractEnlistFuture.mapOnTopolog= y(GridNearTxAbstractEnlistFuture.java:331) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.near.GridNearTxAbstractEnlistFuture.init(GridNea= rTxAbstractEnlistFuture.java:246) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.near.GridNearTxLocal.updateAsync(GridNearTxLocal= .java:2099) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.near.GridNearTxLocal.mvccPutAllAsync0(GridNearTx= Local.java:802) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.near.GridNearTxLocal.putAsync0(GridNearTxLocal.j= ava:582) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.distributed.near.GridNearTxLocal.putAsync(GridNearTxLocal.ja= va:448) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2470) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2468) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4233) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2468) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2449) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2426) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1105) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.ja= va:820) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.datastreamer.DataStreamerCacheUpdaters$Individual.receive(DataStre= amerCacheUpdaters.java:121) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java= :140) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.ut= il.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6817) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.pr= ocessors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.ut= il.worker.GridWorker.run(GridWorker.java:120) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.ut= il.StripedExecutor$Stripe.body(StripedExecutor.java:505) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.ignite.internal.ut= il.worker.GridWorker.run(GridWorker.java:120) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at java.lang.Thread.run(Thread.j= ava:745) -- This message was sent by Atlassian Jira (v8.3.2#803003)