cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dathan Pattishall <datha...@gmail.com>
Subject Re: Ran into an issue where Cassandra Crashed when running out of heap space
Date Tue, 20 Jul 2010 20:17:35 GMT
The storage structure is rather simple.

For every 1 key there is 1 column and a timestamp for that column.

<ColumnFamily Name="Standard2" CompareWith="UTF8Type" />


We don't enable pulling a huge amount of data and all other nodes are
up servicing the same request. I suspect there may be another problem
with Memory management inside Cassandra.

Attaching Jconsole shows that there is a growth of memory and weird
spikes. Unfortunately I did not take a screen shot of the growth of
the spike over time. I'll do that when it occurs again.






On Tue, Jul 20, 2010 at 1:05 PM, Tristan Seligmann
<mithrandi@mithrandi.net> wrote:
> On Tue, Jul 20, 2010 at 9:09 PM, Peter Schuller
> <peter.schuller@infidyne.com> wrote:
>>> CassandraDaemon.java (line 83) Uncaught exception in thread
>>> Thread[pool-1-thread-37895,5,main]
>>> java.lang.OutOfMemoryError: Java heap space
>>>         at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:296)
>>>         at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:203)
>>>         at org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:1116)
>>>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
>>>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>         at java.lang.Thread.run(Thread.java:619)
>>
>> Did someone send garbage on the wrong port, causing thrift to try to
>> read some huge string in the RPC layer? There is a bug filed about
>> this upstream with thrift but I couldn't find it now.
>
> In particular, I've seen this happen when using the wrong protocol
> (framed / unframed) on the client relative to what the server is
> configured for.
> --
> mithrandi, i Ainil en-Balandor, a faer Ambar
>

Mime
View raw message