incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Feng Qu <mail...@gmail.com>
Subject Re: Cassandra 0.8.4 node keeps crashing with OOM errors
Date Wed, 19 Sep 2012 05:43:04 GMT
try increasing vm.max_map_count per http://blog.timstoop.nl/2011/04/20/cassandra-java-io-ioerror-java-io-ioexception-map-failed/  

 
Feng Qu


>________________________________
> From: Raj N <raj.cassandra@gmail.com>
>To: user@cassandra.apache.org 
>Sent: Tuesday, September 18, 2012 6:37 PM
>Subject: Cassandra 0.8.4 node keeps crashing with OOM errors
> 
>
>One of our nodes keeps crashing continuously with out of memory errors. I see the following
error in the logs -
>
>
> INFO 21:03:54,007 Creating new commitlog segment /local3/logs/cassandra/commitlog/CommitLog-1348016634007.log
>Java HotSpot(TM) 64-Bit Server VM warning: Attempt to allocate stack guard pages failed.
>Java HotSpot(TM) 64-Bit Server VM warning: Attempt to allocate stack guard pages failed.
> INFO 21:03:54,017 Replaying /local3/logs/cassandra/commitlog/CommitLog-1347950471775.log,
/local3/logs/cassandra/commitlog/CommitLog-1347952280335.log, /local3/logs/cassandra/commitlog/CommitLog-
>1347954861373.log, /local3/logs/cassandra/commitlog/CommitLog-1347957791957.log, /local3/logs/cassandra/commitlog/CommitLog-1347959168686.log,
/local3/logs/cassandra/commitlog/CommitLog-1347961014
>948.log, /local3/logs/cassandra/commitlog/CommitLog-1347968405068.log, /local3/logs/cassandra/commitlog/CommitLog-1347972420459.log,
/local3/logs/cassandra/commitlog/CommitLog-1347975084979.log, /
>local3/logs/cassandra/commitlog/CommitLog-1347975538081.log, /local3/logs/cassandra/commitlog/CommitLog-1347976033450.log,
/local3/logs/cassandra/commitlog/CommitLog-1347976685447.log, /local3/log
>s/cassandra/commitlog/CommitLog-1347977204225.log, /local3/logs/cassandra/commitlog/CommitLog-1347977904344.log,
/local3/logs/cassandra/commitlog/CommitLog-1347978791835.log, /local3/logs/cassandr
>a/commitlog/CommitLog-1347979595214.log, /local3/logs/cassandra/commitlog/CommitLog-1347980280043.log,
/local3/logs/cassandra/commitlog/CommitLog-1347980822272.log, /local3/logs/cassandra/commitlo
>g/CommitLog-1347981376135.log, /local3/logs/cassandra/commitlog/CommitLog-1347982023403.log,
/local3/logs/cassandra/commitlog/CommitLog-1347982906942.log, /local3/logs/cassandra/commitlog/CommitLo
>g-1347983999163.log, /local3/logs/cassandra/commitlog/CommitLog-1347985475186.log, /local3/logs/cassandra/commitlog/CommitLog-1347987026500.log,
/local3/logs/cassandra/commitlog/CommitLog-13479885
>20038.log, /local3/logs/cassandra/commitlog/CommitLog-1347990191955.log, /local3/logs/cassandra/commitlog/CommitLog-1347991899744.log,
/local3/logs/cassandra/commitlog/CommitLog-1347993488753.log,
> /local3/logs/cassandra/commitlog/CommitLog-1347994801952.log, /local3/logs/cassandra/commitlog/CommitLog-1347995807921.log,
/local3/logs/cassandra/commitlog/CommitLog-1347996959596.log, /local3/l
>ogs/cassandra/commitlog/CommitLog-1347997961248.log, /local3/logs/cassandra/commitlog/CommitLog-1348005812088.log,
/local3/logs/cassandra/commitlog/CommitLog-1348015418757.log
>ERROR 21:03:54,030 Exception encountered during startup.
>java.lang.OutOfMemoryError
>        at java.io.RandomAccessFile.readBytes(Native Method)
>        at java.io.RandomAccessFile.read(RandomAccessFile.java:322)
>        at org.apache.cassandra.io.util.BufferedRandomAccessFile.reBuffer(BufferedRandomAccessFile.java:254)
>        at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:124)
>        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:196)
>        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:158)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:195)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:336)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:91)
>Exception encountered during startup.
>java.lang.OutOfMemoryError
>        at java.io.RandomAccessFile.readBytes(Native Method)
>        at java.io.RandomAccessFile.read(RandomAccessFile.java:322)
>        at org.apache.cassandra.io.util.BufferedRandomAccessFile.reBuffer(BufferedRandomAccessFile.java:254)
>        at org.apache.cassandra.io.util.BufferedRandomAccessFile.<init>(BufferedRandomAccessFile.java:124)
>        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:196)
>        at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:158)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:195)
>        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:336)
>        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:91)
>Java HotSpot(TM) 64-Bit Server VM warning: Attempt to allocate stack guard pages failed.
>
>
>
>
>Has anybody seen this before?
>
>
>-Rajesh
>
>
Mime
View raw message