cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ben Manes (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-5661) Discard pooled readers for cold data
Date Sun, 07 Jul 2013 03:35:53 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701495#comment-13701495
] 

Ben Manes edited comment on CASSANDRA-5661 at 7/7/13 3:35 AM:
--------------------------------------------------------------

"rarely explicitly invalidated" is in regards to a cache, as Jonathan originally described
the problem as a multimap cache instead of as an object pool. He also expressed concern with
evicting a block of buffers at once when he conceived of the same model that you implemented.

I am intimately familiar with Guava's cache as I designed the algorithms, ported and wrote
code for it, and advised on the api. Unfortunately I am not familiar with Cassandra's needs
and its code, so the pool was implemented based on a brief description of the problem and
ideal behavior.

It was a fun exercise for a long weekend. I'd recommend writing tests and benchmarks, which
unfortunately appears to be missing with the patch in its current form. Of couse use whatever
makes the most sense.
                
      was (Author: ben.manes):
    "rarely explicitly invalidated" is regards to a cache, as Jonathan originally described
the problem as a multimap cache instead of as an object pool. He also expressed concern with
evicting a block of buffers at once when he conceived of the same model that you implemented.

I am intimately familiar with Guava's cache as I designed the algorithms, ported and wrote
code for it, and advised on the api. Unfortunately I am not familiar with Cassandra's needs
and its code, so the pool was implemented based on a brief description of the problem and
ideal behavior.

It was a fun exercise for a long weekend. I'd recommend writing tests and benchmarks, which
unfortunately appears to be missing with the patch in its current form. Of couse use whatever
makes the most sense.
                  
> Discard pooled readers for cold data
> ------------------------------------
>
>                 Key: CASSANDRA-5661
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.1
>            Reporter: Jonathan Ellis
>            Assignee: Pavel Yaskevich
>             Fix For: 1.2.7
>
>         Attachments: CASSANDRA-5661.patch, DominatorTree.png, Histogram.png
>
>
> Reader pooling was introduced in CASSANDRA-4942 but pooled RandomAccessReaders are never
cleaned up until the SSTableReader is closed.  So memory use is "the worst case simultaneous
RAR we had open for this file, forever."
> We should introduce a global limit on how much memory to use for RAR, and evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message