cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan LaRocque (JIRA)" <>
Subject [jira] [Updated] (CASSANDRA-2708) memory leak in CompactionManager's estimatedCompactions
Date Wed, 25 May 2011 20:55:47 GMT


Dan LaRocque updated CASSANDRA-2708:

    Comment: was deleted

(was: Adding my 3-line patch against cassandra-0.7 r1127675)

> memory leak in CompactionManager's estimatedCompactions
> -------------------------------------------------------
>                 Key: CASSANDRA-2708
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.6
>            Reporter: Dan LaRocque
>            Priority: Minor
> CompactionManager's estimatedCompactions map seems to hold all or most ColumnFamilyStores
in the system as keys.  Keys are never removed from estimatedCompactions.
> I have a project that embeds Cassandra as a storage backend.  Some of my integration
tests create and drop a single keyspace and pair of column families a hundred or 150 times
in one JVM.  These tests always OOM'd.  Loading some near-death heapdumps in mat suggested
CompactionManager's estimatedCompactions held over 80% of total heap via its ColumnFamilyStore
keys.  estimatedCompactions had the only inbound reference to these CFSs, and the CFSs themselves
had invalid = true.
> As a workaround, I changed estimatedCompactions to a WeakReference-keyed map (using Guava
MapMaker).  My integration tests no longer OOM.
> I'm generally unfamiliar with Cassandra's guts.  I don't know whether weak referencing
the keys of estimatedCompactions is correct (or ideal).  But, that did seem to confirm my
guess that retained references to dead CFSs in estimatedCompactions were swamping my heap
after lots of Keyspace+ColumnFamily drops.

This message is automatically generated by JIRA.
For more information on JIRA, see:

View raw message