Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Thu, 27 Jun 2013 04:03:20 +0000 (UTC)
From: "Pavel Yaskevich (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12653702.1371648677234.178379.1372305800479@arcas>
In-Reply-To: <JIRA.12653702.1371648677234@arcas>
References: <JIRA.12653702.1371648677234@arcas>
Subject: [jira] [Commented] (CASSANDRA-5661) Discard pooled readers for cold
 data
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694466#comment-13694466 ] 

Pavel Yaskevich commented on CASSANDRA-5661:
--------------------------------------------

In other words, my initial idea didn't take into account leveled compaction, it was simple - by paying overall small memory price (one chunk size or less depending on size of file) per 1GB we can minimize GC work, memory allocation and number of syscalls per read. With leveled compaction that strategy doesn't work as price per 1 GB is pretty big as most of the files are very small which increases allocation rate to cache and GC activity by very frequent compactions.
                
> Discard pooled readers for cold data
> ------------------------------------
>
>                 Key: CASSANDRA-5661
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5661
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2.1
>            Reporter: Jonathan Ellis
>            Assignee: Pavel Yaskevich
>             Fix For: 1.2.7
>
>         Attachments: DominatorTree.png, Histogram.png
>
>
> Reader pooling was introduced in CASSANDRA-4942 but pooled RandomAccessReaders are never cleaned up until the SSTableReader is closed.  So memory use is "the worst case simultaneous RAR we had open for this file, forever."
> We should introduce a global limit on how much memory to use for RAR, and evict old ones.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira