distributedlog-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sijie Guo (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DL-101) Improve session expire handling on fetching log segments for readers
Date Tue, 06 Dec 2016 17:04:58 GMT
Sijie Guo created DL-101:

             Summary: Improve session expire handling on fetching log segments for readers
                 Key: DL-101
                 URL: https://issues.apache.org/jira/browse/DL-101
             Project: DistributedLog
          Issue Type: Improvement
          Components: distributedlog-core
            Reporter: Sijie Guo
            Assignee: Sijie Guo

    The log segment management in DL is now done by 3 parts.

    - a LogSegmentMetadataStore (one per namespace instance): it is used for fetching the
log segments from log segment metadata store (ZooKeeper). it doesn't do any caching.
    - a LogSegmentMetadataCache (one per namespace instance): it is a guava cache based metadata
cache. it maintains a mapping between log segment metadata path and the log segment metadata.
it manages the cache for the log segments that will be accessed in this namespace instance.
it doesn't manage the sequence of the log segments for streams.
    - a PerStreamLogSegmentCache for each BKLogHandler. the log segment cache is per stream.
it maintains the sequence of the log segments.

    BKLogWriteHandler doesn't watch the log segment changes. It fetches minimal number of
log segments when it is created and fetches the full list of log segments for truncations.
New log segments will be added to the per stream log segment cache with log segment rolling.

    BKLogReadHandler watch the log segments changes and only notify when the list of log segments
is changed. the session handling which is specific to the metadata store is hidden to the
implementations of LogSegmentMetadataStore.

    The change tries to cleanup bunch of unused methods in BKLog{Read,Write}Handler too.

This message was sent by Atlassian JIRA

View raw message