From issues-return-116494-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Wed Feb 3 09:14:02 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 0FF84180670 for ; Wed, 3 Feb 2021 10:14:02 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 4D3C54461B for ; Wed, 3 Feb 2021 09:14:01 +0000 (UTC) Received: (qmail 58251 invoked by uid 500); 3 Feb 2021 09:14:01 -0000 Mailing-List: contact issues-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list issues@ignite.apache.org Received: (qmail 58225 invoked by uid 99); 3 Feb 2021 09:14:01 -0000 Received: from mailrelay1-he-de.apache.org (HELO mailrelay1-he-de.apache.org) (116.203.21.61) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Feb 2021 09:14:01 +0000 Received: from jira2-he-de.apache.org (unknown [IPv6:2a01:4f8:242:1f49::2]) by mailrelay1-he-de.apache.org (ASF Mail Server at mailrelay1-he-de.apache.org) with ESMTPS id 4AE3F3E824 for ; Wed, 3 Feb 2021 09:14:00 +0000 (UTC) Received: from jira2-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira2-he-de.apache.org (ASF Mail Server at jira2-he-de.apache.org) with ESMTP id 32D64C801E1 for ; Wed, 3 Feb 2021 09:14:00 +0000 (UTC) Date: Wed, 3 Feb 2021 09:14:00 +0000 (UTC) From: "Ignite TC Bot (Jira)" To: issues@ignite.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (IGNITE-13877) Error restarting the node with switching from disabled WAL archiving to enabled MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/IGNITE-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277837#comment-17277837 ] Ignite TC Bot commented on IGNITE-13877: ---------------------------------------- {panel:title=Branch: [pull/8681/head] Base: [master] : Possible Blockers (3)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1} {color:#d04437}PDS 1{color} [[tests 3|https://ci.ignite.apache.org/viewLog.html?buildId=5854816]] * IgnitePdsTestSuite: WalArchiveConsistencyTest.testDecreaseWalSegmentsWitTruncate1[walMode=FSYNC] - New test duration 91s is more that 1 minute * IgnitePdsTestSuite: WalArchiveConsistencyTest.testDecreaseWalSegmentsWitTruncate0[walMode=FSYNC] - New test duration 86s is more that 1 minute * IgnitePdsTestSuite: WalArchiveConsistencyTest.testNotChangeWalSegmentsWitTruncate[walMode=FSYNC] - New test duration 76s is more that 1 minute {panel} {panel:title=Branch: [pull/8681/head] Base: [master] : New Tests (10)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1} {color:#00008b}PDS 1{color} [[tests 10|https://ci.ignite.apache.org/viewLog.html?buildId=5854816]] * {color:#013220}IgnitePdsTestSuite: WalArchiveConsistencyTest.testDecreaseWalSegmentsWithoutTruncate[walMode=LOG_ONLY] - PASSED{color} * {color:#013220}IgnitePdsTestSuite: WalArchiveConsistencyTest.testNotChangeWalSegmentsWitTruncate[walMode=LOG_ONLY] - PASSED{color} * {color:#013220}IgnitePdsTestSuite: WalArchiveConsistencyTest.testDecreaseWalSegmentsWitTruncate1[walMode=LOG_ONLY] - PASSED{color} * {color:#013220}IgnitePdsTestSuite: WalArchiveConsistencyTest.testDecreaseWalSegmentsWitTruncate1[walMode=FSYNC] - PASSED{color} * {color:#013220}IgnitePdsTestSuite: WalArchiveConsistencyTest.testDecreaseWalSegmentsWithoutTruncate[walMode=FSYNC] - PASSED{color} * {color:#013220}IgnitePdsTestSuite: WalArchiveConsistencyTest.testIncreaseWalSegmentsWithoutTruncate[walMode=LOG_ONLY] - PASSED{color} * {color:#013220}IgnitePdsTestSuite: WalArchiveConsistencyTest.testDecreaseWalSegmentsWitTruncate0[walMode=FSYNC] - PASSED{color} * {color:#013220}IgnitePdsTestSuite: WalArchiveConsistencyTest.testDecreaseWalSegmentsWitTruncate0[walMode=LOG_ONLY] - PASSED{color} * {color:#013220}IgnitePdsTestSuite: WalArchiveConsistencyTest.testNotChangeWalSegmentsWitTruncate[walMode=FSYNC] - PASSED{color} * {color:#013220}IgnitePdsTestSuite: WalArchiveConsistencyTest.testIncreaseWalSegmentsWithoutTruncate[walMode=FSYNC] - PASSED{color} {panel} [TeamCity *--> Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=5854845&buildTypeId=IgniteTests24Java8_RunAll] > Error restarting the node with switching from disabled WAL archiving to enabled > ------------------------------------------------------------------------------- > > Key: IGNITE-13877 > URL: https://issues.apache.org/jira/browse/IGNITE-13877 > Project: Ignite > Issue Type: Bug > Components: persistence > Reporter: Kirill Tkalenko > Assignee: Kirill Tkalenko > Priority: Major > Fix For: 2.11 > > Attachments: Ignite13877Test.java > > Time Spent: 1h > Remaining Estimate: 0h > > If a user starts a node with WAL archiving disabled, and then poured data there and there were more than *DataStorageConfiguration#walSegments* and then wants to restart a node with WAL archiving enabled, they will fail due to the following error: > {noformat} > SEVERE: Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.processors.cache.persistence.StorageException: Failed to read checkpoint record from WAL, persistence consistency cannot be guaranteed. Make sure configuration points to correct WAL folders and WAL folder is properly mounted [ptr=FileWALPointer [idx=11, fileOff=15864934, len=21409], walPath=db/wal, walArchive=db/wal/archive]]] > class org.apache.ignite.internal.processors.cache.persistence.StorageException: Failed to read checkpoint record from WAL, persistence consistency cannot be guaranteed. Make sure configuration points to correct WAL folders and WAL folder is properly mounted [ptr=FileWALPointer [idx=11, fileOff=15864934, len=21409], walPath=db/wal, walArchive=db/wal/archive] > at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.performBinaryMemoryRestore(GridCacheDatabaseSharedManager.java:2324) > at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:799) > at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:3523) > at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1206) > at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2089) > at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1758) > at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1147) > at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1065) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:951) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:850) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:720) > at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:689) > at org.apache.ignite.Ignition.start(Ignition.java:344) > {noformat} > At this point, the user can be offered the following workaround: > Move all segments to WAL archive directory (include consistentId directory) as they are except the last one. Last one rename as index % *DataStorageConfiguration#walSegments*. > Described workaround should be done automatically without user intervention. -- This message was sent by Atlassian Jira (v8.3.4#803005)