cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Nickerson <>
Subject Re: Out of Memory Error While Opening SSTables on Startup
Date Tue, 10 Feb 2015 22:04:37 GMT
I was having trouble with snapshots failing while trying to repair that
table (
I have a repair running on it now, and it seems to be going successfully
this time. I am going to wait for that to finish, then try a
manual nodetool compact. If that goes successfully, then would it be safe
to chalk the lack of compaction on this table in the past up to 2.1.2

 ~ Paul Nickerson

On Tue, Feb 10, 2015 at 3:34 PM, Chris Lohfink <> wrote:

> Your cluster is probably having issues with compactions (with STCS you
> should never have this many).  I would probably punt with
> OpsCenter/rollups60. Turn the node off and move all of the sstables off to
> a different directory for backup (or just rm if you really don't care about
> 1 minute metrics), than turn the server back on.
> Once you get your cluster running again go back and investigate why
> compactions stopped, my guess is you hit an exception in past that killed
> your CompactionExecutor and things just built up slowly until you got to
> this point.
> Chris
> On Tue, Feb 10, 2015 at 2:15 PM, Paul Nickerson <> wrote:
>> Thank you Rob. I tried a 12 GiB heap size, and still crashed out. There
>> are 1,617,289 files under OpsCenter/rollups60.
>> Once I downgraded Cassandra to 2.1.1 (apt-get install cassandra=2.1.1), I
>> was able to start up Cassandra OK with the default heap size formula.
>> Now my cluster is running multiple versions of Cassandra. I think I will
>> downgrade the rest to 2.1.1.
>>  ~ Paul Nickerson
>> On Tue, Feb 10, 2015 at 2:05 PM, Robert Coli <>
>> wrote:
>>> On Tue, Feb 10, 2015 at 11:02 AM, Paul Nickerson <>
>>> wrote:
>>>> I am getting an out of memory error why I try to start Cassandra on one
>>>> of my nodes. Cassandra will run for a minute, and then exit without
>>>> outputting any error in the log file. It is happening while SSTableReader
>>>> is opening a couple hundred thousand things.
>>> ...
>>>> Does anyone know how I might get Cassandra on this node running again?
>>>> I'm not very familiar with correctly tuning Java memory parameters, and I'm
>>>> not sure if that's the right solution in this case anyway.
>>> Try running 2.1.1, and/or increasing heap size beyond 8gb.
>>> Are there actually that many SSTables on disk?
>>> =Rob

View raw message