distributedlog-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DL-101) Improve session expire handling on fetching log segments for readers
Date Wed, 28 Dec 2016 07:33:59 GMT

    [ https://issues.apache.org/jira/browse/DL-101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15782290#comment-15782290
] 

Hudson commented on DL-101:
---------------------------

UNSTABLE: Integrated in Jenkins build distributedlog-nightly-build #170 (See [https://builds.apache.org/job/distributedlog-nightly-build/170/])
DL-101: Improve session expire handling on fetching log segments for (sijieg: rev 9ee7d016072f45060d14af3e5e6ddd75a92285f6)
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/AsyncNotification.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/TestReadUtils.java
* (edit) distributedlog-service/src/main/java/com/twitter/distributedlog/service/MonitorService.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/BKDistributedLogNamespace.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/logsegment/LogSegmentMetadataStore.java
* (add) distributedlog-core/src/main/java/com/twitter/distributedlog/function/GetVersionedValueFunction.java
* (delete) distributedlog-core/src/test/java/com/twitter/distributedlog/logsegment/TestLogSegmentCache.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/BKSyncLogReaderDLSN.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/TestBKLogReadHandler.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/TestReadAhead.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/callback/LogSegmentListener.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/DLMTestUtil.java
* (delete) distributedlog-core/src/main/java/com/twitter/distributedlog/logsegment/LogSegmentCache.java
* (add) distributedlog-core/src/main/java/com/twitter/distributedlog/logsegment/PerStreamLogSegmentCache.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/tools/TestDistributedLogTool.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/NonBlockingReadsTestUtil.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/admin/TestDistributedLogAdmin.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/impl/ZKLogSegmentMetadataStore.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/callback/LogSegmentNamesListener.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/impl/TestZKLogSegmentMetadataStore.java
* (edit) distributedlog-protocol/src/main/thrift/service.thrift
* (add) distributedlog-core/src/test/java/com/twitter/distributedlog/logsegment/TestPerStreamLogSegmentCache.java
* (add) distributedlog-protocol/src/main/java/com/twitter/distributedlog/exceptions/LogSegmentNotFoundException.java
* (add) distributedlog-core/src/main/java/com/twitter/distributedlog/function/CloseAsyncCloseableFunction.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/BKLogWriteHandler.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/readahead/ReadAheadWorker.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/TestAsyncReaderWriter.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/DistributedLogConfiguration.java
* (add) distributedlog-core/src/main/java/com/twitter/distributedlog/logsegment/LogSegmentMetadataCache.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/BKAsyncLogReaderDLSN.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/util/FutureUtils.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/BKDistributedLogManager.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/admin/TestDLCK.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/BKLogHandler.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/BKLogReadHandler.java
* (edit) distributedlog-core/src/test/java/com/twitter/distributedlog/TestBKDistributedLogManager.java
* (edit) distributedlog-core/src/main/java/com/twitter/distributedlog/LogSegmentMetadata.java


> Improve session expire handling on fetching log segments for readers
> --------------------------------------------------------------------
>
>                 Key: DL-101
>                 URL: https://issues.apache.org/jira/browse/DL-101
>             Project: DistributedLog
>          Issue Type: Improvement
>          Components: distributedlog-core
>            Reporter: Sijie Guo
>            Assignee: Sijie Guo
>
>     The log segment management in DL is now done by 3 parts.
>     - a LogSegmentMetadataStore (one per namespace instance): it is used for fetching
the log segments from log segment metadata store (ZooKeeper). it doesn't do any caching.
>     - a LogSegmentMetadataCache (one per namespace instance): it is a guava cache based
metadata cache. it maintains a mapping between log segment metadata path and the log segment
metadata. it manages the cache for the log segments that will be accessed in this namespace
instance. it doesn't manage the sequence of the log segments for streams.
>     - a PerStreamLogSegmentCache for each BKLogHandler. the log segment cache is per
stream. it maintains the sequence of the log segments.
>     BKLogWriteHandler doesn't watch the log segment changes. It fetches minimal number
of log segments when it is created and fetches the full list of log segments for truncations.
New log segments will be added to the per stream log segment cache with log segment rolling.
>     BKLogReadHandler watch the log segments changes and only notify when the list of
log segments is changed. the session handling which is specific to the metadata store is hidden
to the implementations of LogSegmentMetadataStore.
>     The change tries to cleanup bunch of unused methods in BKLog{Read,Write}Handler too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message