Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Mon, 14 Nov 2011 14:26:51 +0000 (UTC)
From: "Sylvain Lebresne (Commented) (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: 
 <2126369372.26800.1321280811656.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <172077699.40867.1320069572208.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (CASSANDRA-3427) CompressionMetadata is not
 shared across threads, we create a new one for each read
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13149645#comment-13149645 ] 

Sylvain Lebresne commented on CASSANDRA-3427:
---------------------------------------------

I've attached a tiny patch (0001-debugging.patch) that prints in the log the size of the long array we allocate for the chunk offsets. Would you mind trying with this and attach a log of when startup hits one of the OOM you pasted earlier (feel free to use a 8GB heap if it's easier to reproduce). I'd like to know if those offsets are indeed the problem.
                
> CompressionMetadata is not shared across threads, we create a new one for each read
> -----------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-3427
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3427
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: Sylvain Lebresne
>            Assignee: Sylvain Lebresne
>              Labels: compression
>             Fix For: 1.0.2
>
>         Attachments: 0001-debugging.patch, 3427.patch, 3427_v2.patch, CASSANDRA-3427.patch, jmx_jvm_memory-month.png, jmx_jvm_memory-week.png
>
>
> The CompressionMetada holds the compressed block offsets in memory. Without being absolutely huge, this is still of non-negligible size as soon as you have a bit of data in the DB. Reallocating this for each read is a very bad idea.
> Note that this only affect range queries, since "normal" queries uses CompressedSegmentedFile that does reuse a unique CompressionMetadata instance.
> ( Background: http://thread.gmane.org/gmane.comp.db.cassandra.user/21362 )

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira