Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Sun, 7 Jul 2013 02:25:49 +0000 (UTC)
From: "Jonathan Ellis (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12653702.1371648677234.8156.1373163949077@arcas>
In-Reply-To: <JIRA.12653702.1371648677234@arcas>
References: <JIRA.12653702.1371648677234@arcas>
Subject: [jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold
 data
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701480#comment-13701480 ] 

Jonathan Ellis commented on CASSANDRA-5661:
-------------------------------------------

I talked this over a bit with [~ben.manes] (author of CLHM).  Here's his take:

{quote}
[This] sounds less like a multimap cache than a multi-way object pool. To me a multimap cache would be like any other cache where entries are read frequently, rarely explicitly invalidated, and evicted by a boundary condition. An object pool has instances checked in and out, so care needs be taken to make sure the transfer overhead is cheap. I think you want a more advanced pool with global boundary conditions and is multi-way, so more complex than a traditional database connection pool. For that, actually, a few years ago I advised the author of BoneCP to use a LinkedTransferQueue to leverage elimination to avoid contention which provided the performance improvements to make his library the fastest available.
{quote}

Ben put together an implementation at https://github.com/ben-manes/multiway-pool:

{quote}
 * A concurrent object pool that supports pooling multiple resources that are associated with a
* single key. A resource is borrowed from the pool, used exclusively, and released back for reuse
* by another caller. This implementation can optionally be bounded by a maximum size, time-to-live,
* or time-to-idle policies.
* <p>
* A traditional object pool is homogeneous; all of the resources are identical in the data and
* capabilities offered. For example a database connection pool to a shared database instance. A
* multiway object pool is heterogeneous; resources may differ in the data and capabilities offered.
* For example a flat file database may pool random access readers to the database table files. The
* relationship of a single-way to a multi-way object pool is similar to that of a map to a
* multimap.
* <p>
* When this pool is bounded any resource is eligible for eviction regardless of the key that it is
* associated with. A size based bound will evict resources by a best-effort LRU policy and a time
* based policy will evict by either a time-to-idle and/or time-to-live policy. The resource's life
* cycle can be instrumented, such as when cleaning up after eviction, by using the appropriate
* {@link ResourceLifecycle} method.
{quote}
                
> Discard pooled readers for cold data
> ------------------------------------
>
>                 Key: CASSANDRA-5661
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.1
>            Reporter: Jonathan Ellis
>            Assignee: Pavel Yaskevich
>             Fix For: 1.2.7
>
>         Attachments: CASSANDRA-5661.patch, DominatorTree.png, Histogram.png
>
>
> Reader pooling was introduced in CASSANDRA-4942 but pooled RandomAccessReaders are never cleaned up until the SSTableReader is closed.  So memory use is "the worst case simultaneous RAR we had open for this file, forever."
> We should introduce a global limit on how much memory to use for RAR, and evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira