ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Lang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-6506) Unable to activate cluster node if it was stopped during persistent storage checkpoint
Date Tue, 26 Sep 2017 17:55:00 GMT
Joel Lang created IGNITE-6506:
---------------------------------

             Summary: Unable to activate cluster node if it was stopped during persistent
storage checkpoint
                 Key: IGNITE-6506
                 URL: https://issues.apache.org/jira/browse/IGNITE-6506
             Project: Ignite
          Issue Type: Bug
          Components: persistence
    Affects Versions: 2.2
            Reporter: Joel Lang
            Priority: Critical


I have a cluster with two nodes: A and B.

On startup, node A and B wait for each other to be connected and then node A will attempt
to activate the cluster.

While testing high availability we find that if a node is stopped during the persistent store
checkpoint, we cannot activate the cluster on startup without deleting the persistent storage
directory. Specifically in the case where node A is stopped during checkpointing, upon the
next startup it will encounter several exceptions during activation and then hang without
completing activation.

Here is the log.


{noformat}
2017-09-26 12:11:24 [tcp-disco-msg-worker-#2%mbe%] INFO  o.a.i.i.p.c.GridClusterStateProcessor
- Start state transition: true
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.ignite.internal.exchange.time - Started
exchange init [topVer=AffinityTopologyVersion [topVer=2, minorTopVer=1], crd=true, evt=18,
node=TcpDiscoveryNode [id=62cf0ccb-e376-4b80-8d2d-98115c3a2990, addrs=[10.5.17.19, 127.0.0.1],
sockAddrs=[shouvdevmbe02.petrolink.net/10.5.17.19:47510, /127.0.0.1:47510], discPort=47510,
order=1, intOrder=1, lastExchangeTime=1506445884063, loc=true, ver=2.2.0#20170915-sha1:5747ce6b,
isClient=false], evtNode=TcpDiscoveryNode [id=62cf0ccb-e376-4b80-8d2d-98115c3a2990, addrs=[10.5.17.19,
127.0.0.1], sockAddrs=[shouvdevmbe02.petrolink.net/10.5.17.19:47510, /127.0.0.1:47510], discPort=47510,
order=1, intOrder=1, lastExchangeTime=1506445884063, loc=true, ver=2.2.0#20170915-sha1:5747ce6b,
isClient=false], customEvt=ChangeGlobalStateMessage [id=1d0cb2fbe51-7967bd11-40aa-40fe-b0a6-c43302cd4ee7,
reqId=f7155dea-fede-4340-b244-7a3b65f167a8, initiatingNodeId=62cf0ccb-e376-4b80-8d2d-98115c3a2990,
activate=true]]
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture
- Start activation process [nodeId=62cf0ccb-e376-4b80-8d2d-98115c3a2990, client=false, topVer=AffinityTopologyVersion
[topVer=2, minorTopVer=1]]
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.p.f.FilePageStoreManager
- Resolved page store work directory: /opt/mbe1/ignite/db/mbe_MBE1
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.p.w.FileWriteAheadLogManager
- Resolved write ahead log work directory: /opt/mbe1/ignite/db/wal/mbe_MBE1
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.p.w.FileWriteAheadLogManager
- Resolved write ahead log archive directory: /opt/mbe1/ignite/db/wal/archive/mbe_MBE1
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] WARN  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager
- No user-defined default MemoryPolicy found; system default of 1GB size will be used.
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.p.pagemem.PageMemoryImpl
- Started page memory [memoryAllocated=100.0 MiB, pages=48592, tableSize=2.9 MiB, checkpointBuffer=819.4
MiB]
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.p.pagemem.PageMemoryImpl
- Started page memory [memoryAllocated=3.1 GiB, pages=1544064, tableSize=91.0 MiB, checkpointBuffer=819.4
MiB]
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager
- Read checkpoint status: start marker = /opt/mbe1/ignite/db/mbe_MBE1/cp/1506444061104-38b80aaa-8c3d-4572-a42e-5b7a3b472505-START.bin,
end marker = /opt/mbe1/ignite/db/mbe_MBE1/cp/1506442980839-ff65a0dc-3d83-436a-8329-7b3a31fe5ffc-END.bin
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager
- Checking memory state [lastValidPos=FileWALPointer [idx=139, fileOffset=31406805, len=20731,
forceFlush=false], lastMarked=FileWALPointer [idx=0, fileOffset=0, len=0, forceFlush=false],
lastCheckpointId=38b80aaa-8c3d-4572-a42e-5b7a3b472505]
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] WARN  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager
- Ignite node stopped in the middle of checkpoint. Will restore memory state and finish checkpoint
on node start.
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] ERROR o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture
- Failed to activate node components [nodeId=62cf0ccb-e376-4b80-8d2d-98115c3a2990, client=false,
topVer=AffinityTopologyVersion [topVer=2, minorTopVer=1]]
java.lang.ArrayIndexOutOfBoundsException: -1
	at org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions.forVersion(IOVersions.java:82)
	at org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions.forPage(IOVersions.java:92)
	at org.apache.ignite.internal.pagemem.wal.record.delta.DataPageInsertRecord.applyDelta(DataPageInsertRecord.java:57)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1415)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:539)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClusterStateChangeRequest(GridDhtPartitionsExchangeFuture.java:770)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:574)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1901)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
	at java.lang.Thread.run(Thread.java:745)
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture
- Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=2,
minorTopVer=1], waitTime=0ms]
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager
- Read checkpoint status: start marker = /opt/mbe1/ignite/db/mbe_MBE1/cp/1506444061104-38b80aaa-8c3d-4572-a42e-5b7a3b472505-START.bin,
end marker = /opt/mbe1/ignite/db/mbe_MBE1/cp/1506442980839-ff65a0dc-3d83-436a-8329-7b3a31fe5ffc-END.bin
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.p.GridCacheDatabaseSharedManager
- Applying lost cache updates since last checkpoint record [lastMarked=FileWALPointer [idx=0,
fileOffset=0, len=0, forceFlush=false], lastCheckpointId=38b80aaa-8c3d-4572-a42e-5b7a3b472505]
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] ERROR o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture
- Failed to reinitialize local partitions (preloading will be stopped): GridDhtPartitionExchangeId
[topVer=AffinityTopologyVersion [topVer=2, minorTopVer=1], nodeId=62cf0ccb, evt=DISCOVERY_CUSTOM_EVT]
org.apache.ignite.IgniteCheckedException: WAL history is too short [descs=[org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@5f,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@60,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@61,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@62,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@63,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@64,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@65,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@66,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@67,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@68,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@69,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6a,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6b,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6c,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6d,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6e,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6f,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@70,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@71,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@72,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@73,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@74,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@75,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@76,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@77,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@78,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@79,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7a,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7b,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7c,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7d,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7e,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7f,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@80,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@81,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@82,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@83,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@84,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@85,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@86,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@87,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@88,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@89,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@8a,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@8b],
start=FileWALPointer [idx=0, fileOffset=0, len=0, forceFlush=false]]
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.init(FileWriteAheadLogManager.java:2375)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:2350)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:2290)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:553)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:1484)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:938)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:742)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:994)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:632)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1901)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
	at java.lang.Thread.run(Thread.java:745)
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] INFO  o.a.i.i.p.c.d.d.p.GridDhtPartitionsExchangeFuture
- Snapshot initialization completed [topVer=AffinityTopologyVersion [topVer=2, minorTopVer=1],
time=0ms]
2017-09-26 12:11:24 [exchange-worker-#34%mbe%] ERROR o.a.i.i.p.c.GridCachePartitionExchangeManager
- Failed to wait for completion of partition map exchange (preloading will not start): GridDhtPartitionsExchangeFuture
[dummy=false, forcePreload=false, reassign=false, discoEvt=DiscoveryCustomEvent [customMsg=null,
affTopVer=AffinityTopologyVersion [topVer=2, minorTopVer=1], super=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=62cf0ccb-e376-4b80-8d2d-98115c3a2990, addrs=[10.5.17.19, 127.0.0.1], sockAddrs=[shouvdevmbe02.petrolink.net/10.5.17.19:47510,
/127.0.0.1:47510], discPort=47510, order=1, intOrder=1, lastExchangeTime=1506445884308, loc=true,
ver=2.2.0#20170915-sha1:5747ce6b, isClient=false], topVer=2, nodeId8=62cf0ccb, msg=null, type=DISCOVERY_CUSTOM_EVT,
tstamp=1506445884063]], crd=TcpDiscoveryNode [id=62cf0ccb-e376-4b80-8d2d-98115c3a2990, addrs=[10.5.17.19,
127.0.0.1], sockAddrs=[shouvdevmbe02.petrolink.net/10.5.17.19:47510, /127.0.0.1:47510], discPort=47510,
order=1, intOrder=1, lastExchangeTime=1506445884308, loc=true, ver=2.2.0#20170915-sha1:5747ce6b,
isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=2,
minorTopVer=1], nodeId=62cf0ccb, evt=DISCOVERY_CUSTOM_EVT], added=true, initFut=GridFutureAdapter
[ignoreInterrupts=false, state=DONE, res=false, hash=1495762349], init=false, lastVer=null,
partReleaseFut=GridCompoundFuture [rdc=null, initFlag=1, lsnrCalls=4, done=true, cancelled=false,
err=null, futs=[true, true, true, true]], exchActions=null, affChangeMsg=null, skipPreload=false,
clientOnlyExchange=false, initTs=1506445884073, centralizedAff=false, changeGlobalStateE=java.lang.ArrayIndexOutOfBoundsException:
-1, forcedRebFut=null, done=true, evtLatch=0, remaining=[bd81ae2b-91c0-45f1-b589-e73b7ffb56ee],
super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=class o.a.i.IgniteCheckedException:
WAL history is too short [descs=[o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@5f,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@60, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@61,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@62, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@63,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@64, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@65,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@66, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@67,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@68, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@69,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6a, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6b,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6c, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6d,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6e, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6f,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@70, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@71,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@72, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@73,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@74, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@75,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@76, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@77,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@78, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@79,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7a, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7b,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7c, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7d,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7e, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7f,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@80, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@81,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@82, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@83,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@84, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@85,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@86, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@87,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@88, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@89,
o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@8a, o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@8b],
start=FileWALPointer [idx=0, fileOffset=0, len=0, forceFlush=false]], hash=927462353]]
org.apache.ignite.IgniteCheckedException: WAL history is too short [descs=[org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@5f,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@60,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@61,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@62,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@63,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@64,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@65,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@66,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@67,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@68,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@69,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6a,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6b,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6c,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6d,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6e,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@6f,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@70,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@71,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@72,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@73,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@74,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@75,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@76,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@77,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@78,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@79,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7a,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7b,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7c,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7d,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7e,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@7f,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@80,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@81,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@82,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@83,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@84,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@85,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@86,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@87,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@88,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@89,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@8a,
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDescriptor@8b],
start=FileWALPointer [idx=0, fileOffset=0, len=0, forceFlush=false]]
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.init(FileWriteAheadLogManager.java:2375)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:2350)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:2290)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:553)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:1484)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:938)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:742)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:994)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:632)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1901)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
	at java.lang.Thread.run(Thread.java:745)
{noformat}





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message