ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ignite TC Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-10048) Bounded iteration in standalone WAL iterator with compaction enabled may skip records
Date Tue, 30 Oct 2018 13:30:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-10048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16668693#comment-16668693
] 

Ignite TC Bot commented on IGNITE-10048:
----------------------------------------

{panel:title=Possible Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}PDS (Direct IO) 2{color} [[tests 2|https://ci.ignite.apache.org/viewLog.html?buildId=2200459]]
* IgnitePdsNativeIoTestSuite2: IgniteWalIteratorExceptionDuringReadTest.test - 0,0% fails
in last 100 master runs.

{color:#d04437}ZooKeeper (Discovery) 2{color} [[tests 5|https://ci.ignite.apache.org/viewLog.html?buildId=2200461]]
* ZookeeperDiscoverySpiTestSuite2: GridEventConsumeSelfTest.testEventsByFilter - 1,0% fails
in last 100 master runs.

{color:#d04437}PDS 2{color} [[tests 2|https://ci.ignite.apache.org/viewLog.html?buildId=2200463]]
* IgnitePdsTestSuite2: IgniteWalIteratorExceptionDuringReadTest.test - 0,0% fails in last
100 master runs.

{panel}
[TeamCity Run All Results|http://ci.ignite.apache.org/viewLog.html?buildId=2193103&amp;buildTypeId=IgniteTests24Java8_RunAll]

> Bounded iteration in standalone WAL iterator with compaction enabled may skip records
> -------------------------------------------------------------------------------------
>
>                 Key: IGNITE-10048
>                 URL: https://issues.apache.org/jira/browse/IGNITE-10048
>             Project: Ignite
>          Issue Type: Improvement
>            Reporter: Ivan Rakov
>            Assignee: Ivan Rakov
>            Priority: Major
>             Fix For: 2.8
>
>
> Bounded iteration with non-zero start/end offsets may skip some records in intermediate
segments. Reproducer (wal compaction should be enabled):
> {noformat}
>     /**
>      *
>      */
>     public void testBoundedIterationOverSeveralSegments() throws Exception {
>         walCompactionEnabled = true;
>         IgniteEx ig = (IgniteEx)startGrid();
>         String archiveWalDir = getArchiveWalDirPath(ig);
>         ig.cluster().active(true);
>         IgniteCache<Object, Object> cache = ig.getOrCreateCache(
>             new CacheConfiguration<>().setName("c-n").setAffinity(new RendezvousAffinityFunction(false,
32)));
>         IgniteCacheDatabaseSharedManager sharedMgr = ig.context().cache().context().database();
>         IgniteWriteAheadLogManager walMgr = ig.context().cache().context().wal();
>         WALPointer fromPtr = null;
>         int recordsCnt = WAL_SEGMENT_SIZE / 8 /* record size */ * 5;
>         for (int i = 0; i < recordsCnt; i++) {
>             WALPointer ptr = walMgr.log(new PartitionDestroyRecord(i, i));
>             if (i == 100)
>                 fromPtr = ptr;
>         }
>         assertNotNull(fromPtr);
>         cache.put(1, 1);
>         forceCheckpoint();
>         // Generate WAL segments for filling WAL archive folder.
>         for (int i = 0; i < 2 * ig.configuration().getDataStorageConfiguration().getWalSegments();
i++) {
>             sharedMgr.checkpointReadLock();
>             try {
>                 walMgr.log(new SnapshotRecord(i, false), RolloverType.NEXT_SEGMENT);
>             }
>             finally {
>                 sharedMgr.checkpointReadUnlock();
>             }
>         }
>         cache.put(2, 2);
>         forceCheckpoint();
>         U.sleep(5000);
>         stopGrid();
>         WALIterator it = new IgniteWalIteratorFactory(log)
>             .iterator(new IteratorParametersBuilder().from((FileWALPointer)fromPtr).filesOrDirs(archiveWalDir));
>         TreeSet<Integer> foundCounters = new TreeSet<>();
>         it.forEach(x -> {
>             WALRecord rec = x.get2();
>             if (rec instanceof PartitionDestroyRecord)
>                 foundCounters.add(((WalRecordCacheGroupAware)rec).groupId());
>         });
>         assertEquals(new Integer(100), foundCounters.first());
>         assertEquals(new Integer(recordsCnt - 1), foundCounters.last());
>         assertEquals(recordsCnt - 100, foundCounters.size());
>     }
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message