cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Stepachev <>
Subject Cassandra OOM on repair.
Date Fri, 15 Jul 2011 08:21:00 GMT
Hi all.

Cassandra constantly OOM on repair or compaction. Increasing memory doesn't
help (6G)
I can give more, but I think that this is not a regular situation. Cluster
has 4 nodes. RF=3.
Cassandra version 0.8.1

Ring looks like this:
 Address         DC          Rack        Status State   Load            Owns

   127605887595351923798765477786913079296  datacenter1 rack1       Up     Normal  176.96 GB
25.00%  0  datacenter1 rack1       Up     Normal  178.19 GB
25.00%  42535295865117307932921825928971026432  datacenter1 rack1       Up     Normal  178.26 GB
25.00%  85070591730234615865843651857942052864  datacenter1 rack1       Up     Normal  175.2 GB
 25.00%  127605887595351923798765477786913079296

About schema:
I have big rows (>100k, up to several millions). But as I know, it is normal
for cassandra.
All things work relatively good, until I start long running pre-production
tests. I load
data and after a while (~4hours) cluster begin timeout and them some nodes
die with OOM.
My app retries to send, so after short period all nodes becomes down. Very

But now, I can OOM nodes by simple call nodetool repair.
In logs it is clear, how heap rocketjump to
upper limit.
cfstats shows:
config is:
A question is: does anybody knows, what this means. Why cassandra tries to
something big into memory at once?


View raw message