cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold data
Date Sun, 07 Jul 2013 02:25:49 GMT


Jonathan Ellis commented on CASSANDRA-5661:

I talked this over a bit with [~ben.manes] (author of CLHM).  Here's his take:

[This] sounds less like a multimap cache than a multi-way object pool. To me a multimap cache
would be like any other cache where entries are read frequently, rarely explicitly invalidated,
and evicted by a boundary condition. An object pool has instances checked in and out, so care
needs be taken to make sure the transfer overhead is cheap. I think you want a more advanced
pool with global boundary conditions and is multi-way, so more complex than a traditional
database connection pool. For that, actually, a few years ago I advised the author of BoneCP
to use a LinkedTransferQueue to leverage elimination to avoid contention which provided the
performance improvements to make his library the fastest available.

Ben put together an implementation at

 * A concurrent object pool that supports pooling multiple resources that are associated with
* single key. A resource is borrowed from the pool, used exclusively, and released back for
* by another caller. This implementation can optionally be bounded by a maximum size, time-to-live,
* or time-to-idle policies.
* <p>
* A traditional object pool is homogeneous; all of the resources are identical in the data
* capabilities offered. For example a database connection pool to a shared database instance.
* multiway object pool is heterogeneous; resources may differ in the data and capabilities
* For example a flat file database may pool random access readers to the database table files.
* relationship of a single-way to a multi-way object pool is similar to that of a map to a
* multimap.
* <p>
* When this pool is bounded any resource is eligible for eviction regardless of the key that
it is
* associated with. A size based bound will evict resources by a best-effort LRU policy and
a time
* based policy will evict by either a time-to-idle and/or time-to-live policy. The resource's
* cycle can be instrumented, such as when cleaning up after eviction, by using the appropriate
* {@link ResourceLifecycle} method.
> Discard pooled readers for cold data
> ------------------------------------
>                 Key: CASSANDRA-5661
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.1
>            Reporter: Jonathan Ellis
>            Assignee: Pavel Yaskevich
>             Fix For: 1.2.7
>         Attachments: CASSANDRA-5661.patch, DominatorTree.png, Histogram.png
> Reader pooling was introduced in CASSANDRA-4942 but pooled RandomAccessReaders are never
cleaned up until the SSTableReader is closed.  So memory use is "the worst case simultaneous
RAR we had open for this file, forever."
> We should introduce a global limit on how much memory to use for RAR, and evict old ones.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message