accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ben Manes (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-4177) TinyLFU-based BlockCache
Date Sun, 08 May 2016 06:21:12 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275491#comment-15275491
] 

Ben Manes commented on ACCUMULO-4177:
-------------------------------------

[Druid|http://druid.io/] was recently struck by [JDK-8078490 - Missed submissions in ForkJoinPool|https://bugs.openjdk.java.net/browse/JDK-8078490].
This caused the cache to stop evicting because the asynchronous task was never run due to
a race in the executor. The result was either a memory leak (2.2.6) or halting due to the
back pressure (2.3.0). The solution if you are running on an older JDK8 release is to use
a different executor (e.g. same-thread). This critical bug effected 8u40 - 8u60 (current is
8u92) and broke any FJP usage, such as *CompletableFuture*. I confirmed this fix with Doug
Lea when investigating.

In other news, Cassandra recently adopted Caffeine for its [page cache](https://issues.apache.org/jira/browse/CASSANDRA-5863).
Their analysis of the performance, hit rate, and scan tolerance were positive. I'm hoping
to integrate the eviction policy into their off-heap cache ([OHC|https://github.com/snazy/ohc]),
which uses LRU. That cache is extracted into a library so contributing there might make it
easy for you to benefit as well.

> TinyLFU-based BlockCache
> ------------------------
>
>                 Key: ACCUMULO-4177
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4177
>             Project: Accumulo
>          Issue Type: Improvement
>            Reporter: Ben Manes
>             Fix For: 2.0.0
>
>         Attachments: ACCUMULO-4177.patch
>
>
> [LruBlockCache|https://github.com/apache/accumulo/blob/master/core/src/main/java/org/apache/accumulo/core/file/blockfile/cache/LruBlockCache.java]
appears to be based on HBase's. I currently have a patch being reviewed in [HBASE-15560|https://issues.apache.org/jira/browse/HBASE-15560]
that replaces the pseudo Segmented LRU with the TinyLFU eviction policy. That should allow
the cache to make [better predictions|https://github.com/ben-manes/caffeine/wiki/Efficiency]
based on frequency and recency, such as improved scan resistance. The implementation uses
[Caffeine|https://github.com/ben-manes/caffeine], the successor to Guava's cache, to provide
concurrency and keep the patch small.
> Full details are in the JIRA ticket. I think it should be easy to port if there is interest.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message