Looks like your persistence files are corrupted.
You configured LOG_ONLY WAL mode. It doesn't guarantee survival of OS crushes and power failures.
How did you restart your node?

Denis

пн, 2 июл. 2018 г. в 16:40, NO <727418542@qq.com>:
When I restart the node, I get the following error,
The problem persists after restarting the machine。

==========================================================================
[2018-07-02T21:25:52,932][INFO ][exchange-worker-#190][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=/data3/apache-ignite-persistence/node00-8c6172fa-0543-4b8d-937e-75ac27ba21ff/cp/1530535766680-f62c2aa7-4a26-45ad-b311-5b5e9ddc3f0e-START.bin, endMarker=/data3/apache-ignite-persistence/node00-8c6172fa-0543-4b8d-937e-75ac27ba21ff/cp/1530535612596-2ccb2f7a-9578-44a7-ad29-ff5d6e990ae4-END.bin]
[2018-07-02T21:25:52,933][INFO ][exchange-worker-#190][GridCacheDatabaseSharedManager] Checking memory state [lastValidPos=FileWALPointer [idx=845169, fileOff=32892207, len=7995], lastMarked=FileWALPointer [idx=845199, fileOff=43729777, len=7995], lastCheckpointId=f62c2aa7-4a26-45ad-b311-5b5e9ddc3f0e]
[2018-07-02T21:25:52,933][WARN ][exchange-worker-#190][GridCacheDatabaseSharedManager] Ignite node stopped in the middle of checkpoint. Will restore memory state and finish checkpoint on node start.
[2018-07-02T21:25:52,949][INFO ][grid-nio-worker-tcp-comm-0-#153][TcpCommunicationSpi] Accepted incoming communication connection [locAddr=/10.16.133.187:47100, rmtAddr=/10.16.133.186:22315]
[2018-07-02T21:25:53,131][INFO ][grid-nio-worker-tcp-comm-1-#154][TcpCommunicationSpi] Accepted incoming communication connection [locAddr=/10.16.133.187:47100, rmtAddr=/10.16.133.185:32502]
[2018-07-02T21:25:56,112][ERROR][exchange-worker-#190][GridDhtPartitionsExchangeFuture] Failed to reinitialize local partitions (preloading will be stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4917, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3c06c945-de21-4b7f-8830-344306327643, addrs=[10.16.133.187, 127.0.0.1], sockAddrs=[/127.0.0.1:47500, /10.16.133.187:47500], discPort=47500, order=4917, intOrder=2496, lastExchangeTime=1530537954950, loc=true, ver=2.4.0#20180305-sha1:aa342270, isClient=false], topVer=4917, nodeId8=3c06c945, msg=null, type=NODE_JOINED, tstamp=1530537952291], nodeId=3c06c945, evt=NODE_JOINED]
org.apache.ignite.IgniteCheckedException: Unknown page IO type: 0
    at org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getBPlusIO(PageIO.java:567) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:478) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:438) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.pagemem.wal.record.delta.DataPageInsertFragmentRecord.applyDelta(DataPageInsertFragmentRecord.java:58) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1967) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1827) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:725) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:741) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:626) [ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2337) [ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) [ignite-core-2.4.0.jar:2.4.0]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_45]
[2018-07-02T21:25:56,116][INFO ][exchange-worker-#190][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion [topVer=4917, minorTopVer=0], resVer=null, err=class org.apache.ignite.IgniteCheckedException: Unknown page IO type: 0]
[2018-07-02T21:25:56,117][ERROR][main][IgniteKernal] Got exception while starting (will rollback startup routine).
org.apache.ignite.IgniteCheckedException: Unknown page IO type: 0
    at org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getBPlusIO(PageIO.java:567) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:478) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:438) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.pagemem.wal.record.delta.DataPageInsertFragmentRecord.applyDelta(DataPageInsertFragmentRecord.java:58) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1967) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1827) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:725) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:741) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:626) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2337) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) ~[ignite-core-2.4.0.jar:2.4.0]
    at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_45]
[2018-07-02T21:25:56,120][ERROR][exchange-worker-#190][GridCachePartitionExchangeManager] Failed to wait for completion of partition map exchange (preloading will not start): GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3c06c945-de21-4b7f-8830-344306327643, addrs=[10.16.133.187, 127.0.0.1], sockAddrs=[/127.0.0.1:47500, /10.16.133.187:47500], discPort=47500, order=4917, intOrder=2496, lastExchangeTime=1530537954950, loc=true, ver=2.4.0#20180305-sha1:aa342270, isClient=false], topVer=4917, nodeId8=3c06c945, msg=null, type=NODE_JOINED, tstamp=1530537952291], crd=TcpDiscoveryNode [id=a8bf945b-981c-46f6-baae-b31626755e09, addrs=[10.16.133.188, 127.0.0.1], sockAddrs=[/10.16.133.188:47500, /127.0.0.1:47500], discPort=47500, order=3093, intOrder=1555, lastExchangeTime=1530537952080, loc=false, ver=2.4.0#20180305-sha1:aa342270, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=4917, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3c06c945-de21-4b7f-8830-344306327643, addrs=[10.16.133.187, 127.0.0.1], sockAddrs=[/127.0.0.1:47500, /10.16.133.187:47500], discPort=47500, order=4917, intOrder=2496, lastExchangeTime=1530537954950, loc=true, ver=2.4.0#20180305-sha1:aa342270, isClient=false], topVer=4917, nodeId8=3c06c945, msg=null, type=NODE_JOINED, tstamp=1530537952291], nodeId=3c06c945, evt=NODE_JOINED], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=false, hash=1338265304], init=false, lastVer=null, partReleaseFut=null, exchActions=null, affChangeMsg=null, initTs=1530537952341, centralizedAff=false, changeGlobalStateE=null, done=true, state=SRV, evtLatch=0, remaining=[a8fdbf0a-6956-40ad-b0bd-ca85bc1e3668, f24a14f3-8720-4723-9130-0af8803abd5d, 25e9bb3d-6a64-4463-bcc9-fb413b2fa231, 311efb23-c4ad-49a3-9356-f2423538b319, 7851df66-cbd6-4a94-8acd-5d86437ea4d8, 12aaa4f7-0a3f-4b3f-842f-964837f6cae5, a8bf945b-981c-46f6-baae-b31626755e09, c77aef15-7234-4b16-9bdb-09fc6ce26422, 1a9f815c-8b2a-41c8-af99-70bdf0706e7e], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=class o.a.i.IgniteCheckedException: Unknown page IO type: 0, hash=454207607]]
org.apache.ignite.IgniteCheckedException: Unknown page IO type: 0
    at org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getBPlusIO(PageIO.java:567) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:478) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO.getPageIO(PageIO.java:438) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.pagemem.wal.record.delta.DataPageInsertFragmentRecord.applyDelta(DataPageInsertFragmentRecord.java:58) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1967) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1827) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:725) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:741) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:626) ~[ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2337) [ignite-core-2.4.0.jar:2.4.0]
    at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) [ignite-core-2.4.0.jar:2.4.0]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_45]

==========================================================================

config.xml
============================================================================================
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
        ">
    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
   
        <property name="failureDetectionTimeout" value="60000"/>
       
        <property name="clientFailureDetectionTimeout" value="60000"/>
               
        <property name="segmentationPolicy" value="RESTART_JVM"/>
       
        <!-- <property name="publicThreadPoolSize" value="192"/> -->
       
        <!-- <property name="systemThreadPoolSize" value="192"/> -->

        <property name="dataStreamerThreadPoolSize" value="96"/>
       
        <property name="rebalanceThreadPoolSize" value="4" />
       
        <property name="dataStorageConfiguration">
            <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
                <property name="defaultDataRegionConfiguration">
                    <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                        <property name="name" value="qipu_entity_cache_data_region"/>
       
                        <property name="initialSize" value="#{10L * 1024 * 1024 * 1024}"/>
       
                        <property name="maxSize" value="#{100L * 1024 * 1024 * 1024}"/>
       
                        <property name="persistenceEnabled" value="true"/>
       
                        <property name="metricsEnabled" value="true"/>
       
                        <property name="checkpointPageBufferSize" value="#{1 * 1024 * 1024 * 1024}"/>
                    </bean>
                </property>
       
                <property name="storagePath" value="/data3/apache-ignite-persistence"/>
       
                <property name="walPath" value="/data2/apache-ignite-wal"/>
       
                <property name="walArchivePath" value="/data2/apache-ignite-wal/archive"/>
       
                <property name="walSegmentSize" value="#{64 * 1024 * 1024}"/>
       
                <property name="pageSize" value="#{4 * 1024}"/>
       
                <property name="walSegments" value="#{20}"/>
       
                <property name="walMode" value="LOG_ONLY"/>
       
                <property name="metricsEnabled" value="true"/>

                <property name="writeThrottlingEnabled" value="false"/>
               
                <property name="checkpointThreads" value="8"/>
               
                <property name="walThreadLocalBufferSize" value="#{1 * 1024 * 1024}"/>
            </bean>
        </property>
               
        <property name="cacheConfiguration">
            <bean class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="dataRegionName" value="qipu_entity_cache_data_region"/>

                <property name="name" value="qipu_entity_cache"/>

                <property name="cacheMode" value="PARTITIONED"/>

                <property name="partitionLossPolicy" value="IGNORE"/>

                <property name="atomicityMode" value="ATOMIC"/>

                <property name="backups" value="2"/>

                <property name="writeSynchronizationMode" value="FULL_SYNC"/>

                <property name="statisticsEnabled" value="true"/>

                <property name="rebalanceBatchSize" value="#{20 * 1024 * 1024}"/>

                <property name="rebalanceThrottle" value="0"/>
               
                <property name="rebalanceMode" value="ASYNC"/>
           
                <property name="rebalanceBatchesPrefetchCount" value="4"/>
               
                <property name="rebalanceTimeout" value="20000"/>
               
                <property name="maxConcurrentAsyncOperations" value="#{4 * 500}"/>
            </bean>
        </property>
       
        <property name="communicationSpi">
            <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
                <property name="socketWriteTimeout" value="6000"/>
                <property name="messageQueueLimit" value="102400"/>
            </bean>
        </property>
           
        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="forceServerMode" value="true"/>
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses">
                            <list>
                                <!-- In distributed environment, replace with actual host IP address. -->
                                <value>10.16.133.179:47500..47509</value>
                                <value>10.16.133.180:47500..47509</value>
                                <value>10.16.133.181:47500..47509</value>
                                <value>10.16.133.182:47500..47509</value>
                                <value>10.16.133.183:47500..47509</value>
                                <value>10.16.133.184:47500..47509</value>
                                <value>10.16.133.185:47500..47509</value>
                                <value>10.16.133.186:47500..47509</value>
                                <value>10.16.133.187:47500..47509</value>
                                <value>10.16.133.188:47500..47509</value>
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
        <property name="gridLogger">
            <bean class="org.apache.ignite.logger.log4j2.Log4J2Logger">
                <constructor-arg type="java.lang.String" value="/config/ignite-log4j2.xml"/>
            </bean>
        </property>
    </bean>
</beans>

============================================================================================

The current node is unable to provide services. Please help check where the problem is. thank you very much。