cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robbie Strickland (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-10449) OOM on bootstrap due to long GC pause
Date Tue, 06 Oct 2015 00:42:26 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-10449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Robbie Strickland updated CASSANDRA-10449:
------------------------------------------
    Description: 
I have a 20-node cluster with vnodes (default of 256) and 500-700GB per node.  SSTable counts
are <10 per table.  I am attempting to provision additional nodes, but bootstrapping OOMs
every time after about 10 hours with a sudden long GC pause:

{noformat}
INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old Generation GC
in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
...
ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 CassandraDaemon.java:223 - Exception
in thread Thread[MemtableFlushWriter:454,5,main]
java.lang.OutOfMemoryError: Java heap space
{noformat}

I have tried increasing max heap to 48G just to get through the bootstrap, to no avail.

  was:
I have a 20-node cluster with vnodes (default of 256) and 500-700GB per node.  SSTable counts
are <10 per node.  I am attempting to provision additional nodes, but bootstrapping OOMs
every time after about 10 hours with a sudden long GC pause:

{noformat}
INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old Generation GC
in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
...
ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 CassandraDaemon.java:223 - Exception
in thread Thread[MemtableFlushWriter:454,5,main]
java.lang.OutOfMemoryError: Java heap space
{noformat}

I have tried increasing max heap to 48G just to get through the bootstrap, to no avail.


> OOM on bootstrap due to long GC pause
> -------------------------------------
>
>                 Key: CASSANDRA-10449
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10449
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: Ubuntu 14.04
>            Reporter: Robbie Strickland
>              Labels: gc
>
> I have a 20-node cluster with vnodes (default of 256) and 500-700GB per node.  SSTable
counts are <10 per table.  I am attempting to provision additional nodes, but bootstrapping
OOMs every time after about 10 hours with a sudden long GC pause:
> {noformat}
> INFO  [Service Thread] 2015-10-05 23:33:33,373 GCInspector.java:252 - G1 Old Generation
GC in 1586126ms.  G1 Old Gen: 49213756976 -> 49072277176;
> ...
> ERROR [MemtableFlushWriter:454] 2015-10-05 23:33:33,380 CassandraDaemon.java:223 - Exception
in thread Thread[MemtableFlushWriter:454,5,main]
> java.lang.OutOfMemoryError: Java heap space
> {noformat}
> I have tried increasing max heap to 48G just to get through the bootstrap, to no avail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message