jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francesco Mari (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OAK-8006) SegmentBlob#readLongBlobId might cause SegmentNotFoundException on standby
Date Tue, 29 Jan 2019 10:14:00 GMT

    [ https://issues.apache.org/jira/browse/OAK-8006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754821#comment-16754821
] 

Francesco Mari commented on OAK-8006:
-------------------------------------

[~dulceanu], I was able to sketch out a test case that activates the desired code path in
{{SegmentBlob}}. The test case fails to test anything, but if you descend into the call {{blob.length()}}
with your debugger, you can see that the new code is called. Maybe we can build on top of
that?

{noformat}
public class SegmentBlobTest {

    // `BLOB_SIZE` has to be chosen in such a way that is both:
    //
    // - Less than or equal to `minRecordLength` in the Blob Store. This is
    //   necessary in order to create a sufficiently long blob ID.
    // - Greater than or equal to `Segment.BLOB_ID_SMALL_LIMIT` in order to save
    //   the blob ID as a large, external blob ID.
    // - Greater than or equal to `Segment.MEDIUM_LIMIT`, otherwise the content
    //   of the binary will be written as a mere value record instead of a
    //   binary ID referencing an external binary.
    //
    // Since `Segment.MEDIUM_LIMIT` is the largest of the constants above, it is
    // sufficient to set `BLOB_SIZE` to `Segment.MEDIUM_LIMIT`. 

    private static final int BLOB_SIZE = 2 * SegmentTestConstants.MEDIUM_LIMIT;

    private TemporaryFolder folder = new TemporaryFolder(new File("target"));

    private TemporaryBlobStore blobStore = new TemporaryBlobStore(folder) {

        @Override
        protected void configureDataStore(FileDataStore dataStore) {
            dataStore.setMinRecordLength(BLOB_SIZE);
        }

    };

    private TemporaryFileStore fileStore = new TemporaryFileStore(folder, blobStore, false);

    @Rule
    public RuleChain ruleChain = RuleChain.outerRule(folder)
        .around(blobStore)
        .around(fileStore);

    @Test
    public void testReadLongBlobId() throws Exception {
        SegmentNodeStore nodeStore = SegmentNodeStoreBuilders.builder(fileStore.fileStore()).build();
        Blob blob = nodeStore.createBlob(new NullInputStream(BLOB_SIZE));
        blob.length();
    }
}
{noformat}

> SegmentBlob#readLongBlobId might cause SegmentNotFoundException on standby
> --------------------------------------------------------------------------
>
>                 Key: OAK-8006
>                 URL: https://issues.apache.org/jira/browse/OAK-8006
>             Project: Jackrabbit Oak
>          Issue Type: Bug
>          Components: segment-tar, tarmk-standby
>    Affects Versions: 1.6.0
>            Reporter: Andrei Dulceanu
>            Assignee: Andrei Dulceanu
>            Priority: Major
>              Labels: cold-standby
>             Fix For: 1.12, 1.10.1, 1.8.12
>
>         Attachments: OAK-8006.patch
>
>
> When persisting a segment transferred from master, among others, the cold standby needs
to read the binary references from the segment. While this usually doesn't involve any additional
reads from any other segments, there is a special case concerning binary IDs larger than
4092 bytes. These can live in other segments (which got transferred prior to the current segment
and are already on the standby), but it might also be the case that the binary ID is stored
in the same segment. If this happens, the call to {{blobId.getSegment()}}[0], triggers
a new read of the current, un-persisted segment . Thus, a {{SegmentNotFoundException}} is
thrown:
> {noformat}
> 22.01.2019 09:35:59.345 *ERROR* [standby-run-1] org.apache.jackrabbit.oak.segment.standby.client.StandbyClientSync
Failed synchronizing state.
> org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment d40a9da6-06a2-4dc0-ab91-5554a33c02b0
not found
>         at org.apache.jackrabbit.oak.segment.file.AbstractFileStore.readSegmentUncached(AbstractFileStore.java:284)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.file.FileStore.lambda$readSegment$10(FileStore.java:498)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.SegmentCache$NonEmptyCache.lambda$getSegment$0(SegmentCache.java:163)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4724)
[com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
>         at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3522)
[com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
>         at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2315)
[com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
>         at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2278)
[com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
>         at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2193) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
>         at com.google.common.cache.LocalCache.get(LocalCache.java:3932) [com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
>         at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4721)
[com.adobe.granite.osgi.wrapper.guava:15.0.0.0002]
>         at org.apache.jackrabbit.oak.segment.SegmentCache$NonEmptyCache.getSegment(SegmentCache.java:160)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:498)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:153)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.RecordId.getSegment(RecordId.java:98) [org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.SegmentBlob.readLongBlobId(SegmentBlob.java:206)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.SegmentBlob.readBlobId(SegmentBlob.java:163)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.file.AbstractFileStore$3.consume(AbstractFileStore.java:262)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.Segment.forEachRecord(Segment.java:601)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.file.AbstractFileStore.readBinaryReferences(AbstractFileStore.java:257)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.file.FileStore.writeSegment(FileStore.java:533)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.standby.client.StandbyClientSyncExecution.copySegmentFromPrimary(StandbyClientSyncExecution.java:225)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.standby.client.StandbyClientSyncExecution.copySegmentHierarchyFromPrimary(StandbyClientSyncExecution.java:194)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.standby.client.StandbyClientSyncExecution.compareAgainstBaseState(StandbyClientSyncExecution.java:101)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.standby.client.StandbyClientSyncExecution.execute(StandbyClientSyncExecution.java:76)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.jackrabbit.oak.segment.standby.client.StandbyClientSync.run(StandbyClientSync.java:165)
[org.apache.jackrabbit.oak-segment-tar:1.10.0]
>         at org.apache.sling.commons.scheduler.impl.QuartzJobExecutor.execute(QuartzJobExecutor.java:347)
[org.apache.sling.commons.scheduler:2.7.2]
>         at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [org.apache.sling.commons.scheduler:2.7.2]
>         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>         at java.base/java.lang.Thread.run(Thread.java:834){noformat}
>  
> [0] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/SegmentBlob.java#L205



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message