cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data
Date Sun, 07 Jul 2013 02:25:49 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701480#comment-13701480
] 

Jonathan Ellis commented on CASSANDRA-5661:
-------------------------------------------

I talked this over a bit with [~ben.manes] (author of CLHM).  Here's his take:

{quote}
[This] sounds less like a multimap cache than a multi-way object pool. To me a multimap cache
would be like any other cache where entries are read frequently, rarely explicitly invalidated,
and evicted by a boundary condition. An object pool has instances checked in and out, so care
needs be taken to make sure the transfer overhead is cheap. I think you want a more advanced
pool with global boundary conditions and is multi-way, so more complex than a traditional
database connection pool. For that, actually, a few years ago I advised the author of BoneCP
to use a LinkedTransferQueue to leverage elimination to avoid contention which provided the
performance improvements to make his library the fastest available.
{quote}

Ben put together an implementation at https://github.com/ben-manes/multiway-pool:

{quote}
 * A concurrent object pool that supports pooling multiple resources that are associated with
a
* single key. A resource is borrowed from the pool, used exclusively, and released back for
reuse
* by another caller. This implementation can optionally be bounded by a maximum size, time-to-live,
* or time-to-idle policies.
* <p>
* A traditional object pool is homogeneous; all of the resources are identical in the data
and
* capabilities offered. For example a database connection pool to a shared database instance.
A
* multiway object pool is heterogeneous; resources may differ in the data and capabilities
offered.
* For example a flat file database may pool random access readers to the database table files.
The
* relationship of a single-way to a multi-way object pool is similar to that of a map to a
* multimap.
* <p>
* When this pool is bounded any resource is eligible for eviction regardless of the key that
it is
* associated with. A size based bound will evict resources by a best-effort LRU policy and
a time
* based policy will evict by either a time-to-idle and/or time-to-live policy. The resource's
life
* cycle can be instrumented, such as when cleaning up after eviction, by using the appropriate
* {@link ResourceLifecycle} method.
{quote}
                
> Discard pooled readers for cold data
> ------------------------------------
>
>                 Key: CASSANDRA-5661
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.1
>            Reporter: Jonathan Ellis
>            Assignee: Pavel Yaskevich
>             Fix For: 1.2.7
>
>         Attachments: CASSANDRA-5661.patch, DominatorTree.png, Histogram.png
>
>
> Reader pooling was introduced in CASSANDRA-4942 but pooled RandomAccessReaders are never
cleaned up until the SSTableReader is closed.  So memory use is "the worst case simultaneous
RAR we had open for this file, forever."
> We should introduce a global limit on how much memory to use for RAR, and evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message