cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas van Neerijnen <...@bossastudios.com>
Subject Why does a large compaction on one node affect the entire cluster?
Date Thu, 24 May 2012 18:23:51 GMT
Hi all

I am running Cassandra 1.0.10 installed from the apache debs on ubuntu
11.10 on a 7 node cluster.

I moved some tokens around my cluster and now have one node compacting a
large Leveled compaction column family. It has done about 5k out of 10k
outstanding compactions today. The other nodes have all finished.

The weird thing is when it hits a big-ish chunk to compact, for example:
pending tasks: 4555
          compaction type        keyspace   column family bytes
compacted     bytes total  progress
               Compaction          Player    PlayerDetail
213097      4286616517     0.00%
, I see heap usage on it AND all other nodes go insane.
Normal operation on all nodes is a leisurely saw-toothed climb to a CMS at
just below 3/4 heap size every 10 minutes or so.
During the big-ish compaction all nodes in the cluster CMS multiple times
in a minute, with the peaks getting close to heap size.

So my question is why does one node compacting put so much memory pressure
on all the other nodes in the cluster and ruin my day?

Mime
View raw message