incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom van den Berge <...@drillster.com>
Subject Re: OOM while reading key cache
Date Wed, 13 Nov 2013 08:35:05 GMT
I'm having the same problem, after upgrading from 1.2.3 to 1.2.10.

I can remember this was a bug that was solved in the 1.0 or 1.1 version
some time ago, but apparently it got back.
A workaround is to delete the contents of the saved_caches directory before
starting up.


Tom


On Tue, Nov 12, 2013 at 5:15 AM, Aaron Morton <aaron@thelastpickle.com>wrote:

> -6 machines which 8gb RAM each and three 150GB disks each
> -default heap configuration
>
> With 8GB the default heap is 2GB, try kicking that up to 4GB and a 600 to
> 800 MB new heap.
>
> I would guess for the data load  you have 2GB is not enough.
>
> hope that helps.
>
> -----------------
> Aaron Morton
> New Zealand
> @aaronmorton
>
> Co-Founder & Principal Consultant
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>
> On 8/11/2013, at 11:31 pm, olek.stasiak@gmail.com wrote:
>
> Hello,
> I'm facing OOM on reading key_cache
> Cluster conf is as follows:
> -6 machines which 8gb RAM each and three 150GB disks each
> -default heap configuration
> -deafult key cache configuration
> -the biggest keyspace has abt 500GB size (RF: 2, so in fact there is
> 250GB of raw data).
>
> After upgrading first of the machines from 1.2.11 to 2.0.2 i've recieved
> error:
> INFO [main] 2013-11-08 10:53:16,716 AutoSavingCache.java (line 114)
> reading saved cache
>
> /home/synat/nosql_filesystem/cassandra/data/saved_caches/production_storage-METADATA-KeyCache-b.db
> ERROR [main] 2013-11-08 10:53:16,895 CassandraDaemon.java (line 478)
> Exception encountered during startup
> java.lang.OutOfMemoryError: Java heap space
>        at
> org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:394)
>        at
> org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
>        at
> org.apache.cassandra.service.CacheService$KeyCacheSerializer.deserialize(CacheService.java:352)
>        at
> org.apache.cassandra.cache.AutoSavingCache.loadSaved(AutoSavingCache.java:119)
>        at
> org.apache.cassandra.db.ColumnFamilyStore.<init>(ColumnFamilyStore.java:264)
>        at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:409)
>        at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:381)
>        at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:314)
>        at org.apache.cassandra.db.Keyspace.<init>(Keyspace.java:268)
>        at org.apache.cassandra.db.Keyspace.open(Keyspace.java:110)
>        at org.apache.cassandra.db.Keyspace.open(Keyspace.java:88)
>        at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:274)
>        at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:461)
>        at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:504)
>
>
> Error appears every start, so I've decided to disable key cache (this
> was not helpful) and temporarily moved key cache out of cache folder
> (file was of size 13M). That helps in starting node, but this is only
> workaround and it's not demanded configuration. Anyone has any idea
> what is the real cause of problem with oom?
> best regards
> Aleksander
> ps. I've still 5 nodes to upgrade, I'll inform if problem apperas on the
> rest.
>
>
>

Mime
View raw message