There is a limit on the size of the commit log and on how long Hints are stored for. 

I'm not sure why your load was different, I think it was left of hints and commit log. But it's not always easy to diagnose thingsvia email. 

Hopefully nodetool drain or deleting the rest system and starting again will get you moving forwards again.

Aaron Morton
Freelance Cassandra Developer
New Zealand


On 14/12/2012, at 12:50 AM, Sergey Olefir <> wrote:

I'll try nodetool drain, thanks.

But more generally -- are you basically saying that I should not worry about
these things? Data will not keep accumulating indefinitely in production and
it'll not affect performance negatively (despite vast differences in node

Best regards,

aaron morton wrote
try nodetool drain. It will flush everything to disk and the commit log
will be truncated.

HH can be ignored. If you really want them gone they can be purged using
the JMX interface, or you can stop the node and delete the sstables.


Aaron Morton
Freelance Cassandra Developer
New Zealand


On 13/12/2012, at 10:35 AM, Sergey Olefir &lt;


&gt; wrote:

Nick Bailey-2 wrote
Dropping a keyspace causes a snapshot to be taken of the keyspace before
is removed from the schema. So it won't actually delete any data. You
manually delete the data from /var/lib/cassandra/


Indeed, it looks like snapshot is on the file system. However it looks
it is not the only thing by a long shot, i.e.:
cassa1-1:/var/log/cassandra# du -k /spool1/cassandra/data/1.1/
375384  /spool1/cassandra/data/1.1/rainmanLoadTestKeyspace
4       /spool1/cassandra/data/1.1/system/Versions
52      /spool1/cassandra/data/1.1/system/schema_columns
4       /spool1/cassandra/data/1.1/system/Schema
28      /spool1/cassandra/data/1.1/system/NodeIdInfo
4       /spool1/cassandra/data/1.1/system/Migrations
28      /spool1/cassandra/data/1.1/system/schema_keyspaces
28      /spool1/cassandra/data/1.1/system/schema_columnfamilies
786348  /spool1/cassandra/data/1.1/system/HintsColumnFamily
52      /spool1/cassandra/data/1.1/system/LocationInfo
4       /spool1/cassandra/data/1.1/system/IndexInfo
786556  /spool1/cassandra/data/1.1/system
1161944 /spool1/cassandra/data/1.1/

And also 700+MB in the commitlog. Neither of which seemed to 'go away' on
its own when idle or even after running nodetool repair/cleanup and even
dropping keyspace.

I suppose these hints and commitlog may be the reason behind huge
in load on nodes -- but why does it happen and more importantly is it
harmful? Will it keep accumulating?

View this message in context:
Sent from the


mailing list archive at

View this message in context:
Sent from the mailing list archive at