cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Donald Smith <>
Subject Logs of commitlog files
Date Mon, 14 Apr 2014 19:04:02 GMT
1. With cassandra 2.0.6, we have 547G of files in /var/lib/commitlog/.  I started a "nodetool
flush" 65 minutes ago; it's still running.  The 17536 commitlog files have been created in
the last 3 days.  (The node has 2.1T of sstables data in /var/lib/cassandra/data/.  This is
in staging, not prod.) Why so many commit logs?  Here are our commitlog-related settings in

commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
# The size of the individual commitlog file segments.  A commitlog
# archiving commitlog segments (see,
commitlog_segment_size_in_mb: 32
# Total space to use for commitlogs.  Since commitlog segments are
# segment and remove it.  So a small total commitlog space will tend
# commitlog_total_space_in_mb: 4096

Maybe we should set commitlog_total_space_in_mb to something other than the default. According
to OpsCenter, commitlog_total_space_in_mb is "None".    But it seems odd that there'd be so
many commit logs.

The node is under heavy write load.   There are about 2900 compactions pending.

We are NOT archiving commitlogs, via

BTW, the documentation for nodetool<> says:

Flushes memtables (in memory) to SSTables (on disk), which also enables CommitLog<>
segments to be deleted.
But even after doing a flush, the /var/lib/commitlog dir still has 1G of files, even after
waiting 30  minutes.  Each file is 32M in size, plus or minus a few bytes.  I tried this on
other clusters, with much smaller amounts of data.   Even restarting Cassandra doesn't help.

I surmise that the 1GB of commit logs are normal: they probably allocate that space as a workspace.

Thanks,  Don

Donald A. Smith | Senior Software Engineer
P: 425.201.3900 x 3866
C: (206) 819-5965
F: (646) 443-2333<>


View raw message