cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sébastien Giroux (JIRA) <j...@apache.org>
Subject [jira] Commented: (CASSANDRA-2273) Possible Memory leak
Date Fri, 04 Mar 2011 19:03:37 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002783#comment-13002783
] 

Sébastien Giroux commented on CASSANDRA-2273:
---------------------------------------------

And here is one more from a different node:

ERROR 18:51:36,970 Fatal exception in thread Thread[HintedHandoff:1,1,main]
java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:636)
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError:
Java heap space
        at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:356)
        at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:88)
        at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:409)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        ... 3 more
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap
space
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
        at java.util.concurrent.FutureTask.get(FutureTask.java:111)
        at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:352)
        ... 6 more
Caused by: java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:121)
        at org.apache.cassandra.io.sstable.SSTableWriter.<init>(SSTableWriter.java:78)
        at org.apache.cassandra.db.ColumnFamilyStore.createCompactionWriter(ColumnFamilyStore.java:2145)
        at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:448)
        at org.apache.cassandra.db.CompactionManager$4.call(CompactionManager.java:240)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        ... 3 more
ERROR 18:51:36,971 Fatal exception in thread Thread[FlushWriter:1,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:121)
        at org.apache.cassandra.io.sstable.SSTableWriter.<init>(SSTableWriter.java:78)
        at org.apache.cassandra.db.ColumnFamilyStore.createFlushWriter(ColumnFamilyStore.java:2140)
        at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
        at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
        at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:636)


> Possible Memory leak
> --------------------
>
>                 Key: CASSANDRA-2273
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2273
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Sébastien Giroux
>            Priority: Critical
>             Fix For: 0.7.4
>
>         Attachments: heap_peak_OOM.PNG, jconsole_OOM.PNG
>
>
> I have a few problematic nodes in my cluster that will crash OutOfMemory very often.
This is Cassandra 0.7.3 downloaded from Hudson.
> Heap size is 6GB, server memory is 8GB.
> Memtable are flushed at 64MB, I have 5 CFs.
> FlushLargestMemtablesAt is set at 0.8 but doesn't help with this issue.
> I will attach a screenshot showing my issue. There is no compaction going on when the
heap usage start increasing like crazy.
> It could be a configuration issue but it kinda looks like a bug to me.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message