cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-2708) memory leak in CompactionManager's estimatedCompactions
Date Mon, 22 Aug 2011 21:10:29 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis updated CASSANDRA-2708:
--------------------------------------

             Reviewer: jbellis
    Affects Version/s:     (was: 0.7.6)
                       0.7.0
        Fix Version/s: 0.7.9
             Assignee: Dan LaRocque

Sorry Dan, I missed this when you first submitted it.  Committed for 0.7.9.  Thanks for the
help!

> memory leak in CompactionManager's estimatedCompactions
> -------------------------------------------------------
>
>                 Key: CASSANDRA-2708
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2708
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0
>            Reporter: Dan LaRocque
>            Assignee: Dan LaRocque
>            Priority: Minor
>             Fix For: 0.7.9
>
>         Attachments: cassandra-0.7-2708.txt
>
>
> CompactionManager's estimatedCompactions map seems to hold all or most ColumnFamilyStores
in the system as keys.  Keys are never removed from estimatedCompactions.
> I have a project that embeds Cassandra as a storage backend.  Some of my integration
tests create and drop a single keyspace and pair of column families a hundred or 150 times
in one JVM.  These tests always OOM'd.  Loading some near-death heapdumps in mat suggested
CompactionManager's estimatedCompactions held over 80% of total heap via its ColumnFamilyStore
keys.  estimatedCompactions had the only inbound reference to these CFSs, and the CFSs themselves
had invalid = true.
> As a workaround, I changed estimatedCompactions to a WeakReference-keyed map (using Guava
MapMaker).  My integration tests no longer OOM.
> I'm generally unfamiliar with Cassandra's guts.  I don't know whether weak referencing
the keys of estimatedCompactions is correct (or ideal).  But, that did seem to confirm my
guess that retained references to dead CFSs in estimatedCompactions were swamping my heap
after lots of Keyspace+ColumnFamily drops.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message