cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jji...@apache.org
Subject cassandra git commit: Updated commitlog documentation storage_engine.rst
Date Wed, 05 Apr 2017 16:45:12 GMT
Repository: cassandra
Updated Branches:
  refs/heads/trunk 2ba06310d -> 22acb0023


Updated commitlog documentation storage_engine.rst

Patch by Hau Phan; Reviewed by Jon Haddad and Evan Prothro for CASSANDRA-12805


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/22acb002
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/22acb002
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/22acb002

Branch: refs/heads/trunk
Commit: 22acb00235ee081d3555cb1ff2780805e0268b07
Parents: 2ba0631
Author: nothau <hau@rocketmobile.co>
Authored: Thu Oct 27 16:20:32 2016 -0500
Committer: Jeff Jirsa <jeff@jeffjirsa.net>
Committed: Wed Apr 5 09:44:20 2017 -0700

----------------------------------------------------------------------
 doc/source/architecture/storage_engine.rst | 49 ++++++++++++++++++++++++-
 1 file changed, 48 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/22acb002/doc/source/architecture/storage_engine.rst
----------------------------------------------------------------------
diff --git a/doc/source/architecture/storage_engine.rst b/doc/source/architecture/storage_engine.rst
index e4114e5..72d5802 100644
--- a/doc/source/architecture/storage_engine.rst
+++ b/doc/source/architecture/storage_engine.rst
@@ -22,7 +22,54 @@ Storage Engine
 CommitLog
 ^^^^^^^^^
 
-.. todo:: todo
+Commitlogs are an append only log of all mutations local to a Cassandra node. Any data written
to Cassandra will first be written to a commit log before being written to a memtable. This
provides durability in the case of unexpected shutdown. On startup, any mutations in the commit
log will be applied to memtables.
+
+All mutations write optimized by storing in commitlog segments, reducing the number of seeks
needed to write to disk. Commitlog Segments are limited by the "commitlog_segment_size_in_mb"
option, once the size is reached, a new commitlog segment is created. Commitlog segments can
be archived, deleted, or recycled once all its data has been flushed to SSTables.  Commitlog
segments are truncated when Cassandra has written data older than a certain point to the SSTables.
Running "nodetool drain" before stopping Cassandra will write everything in the memtables
to SSTables and remove the need to sync with the commitlogs on startup.
+
+- ``commitlog_segment_size_in_mb``: The default size is 32, which is almost always fine,
but if you are archiving commitlog segments (see commitlog_archiving.properties), then you
probably want a finer granularity of archiving; 8 or 16 MB is reasonable. Max mutation size
is also configurable via max_mutation_size_in_kb setting in cassandra.yaml. The default is
half the size commitlog_segment_size_in_mb * 1024.
+
+***NOTE: If max_mutation_size_in_kb is set explicitly then commitlog_segment_size_in_mb must
be set to at least twice the size of max_mutation_size_in_kb / 1024***
+
+*Default Value:* 32
+
+Commitlogs are an append only log of all mutations local to a Cassandra node. Any data written
to Cassandra will first be written to a commit log before being written to a memtable. This
provides durability in the case of unexpected shutdown. On startup, any mutations in the commit
log will be applied.
+
+- ``commitlog_sync``: may be either “periodic” or “batch.”
+
+  - ``batch``: In batch mode, Cassandra won’t ack writes until the commit log has been
fsynced to disk. It will wait "commitlog_sync_batch_window_in_ms" milliseconds between fsyncs.
This window should be kept short because the writer threads will be unable to do extra work
while waiting. You may need to increase concurrent_writes for the same reason.
+
+    - ``commitlog_sync_batch_window_in_ms``: Time to wait between "batch" fsyncs
+    *Default Value:* 2
+
+  - ``periodic``: In periodic mode, writes are immediately ack'ed, and the CommitLog is simply
synced every "commitlog_sync_period_in_ms" milliseconds.
+
+    - ``commitlog_sync_period_in_ms``: Time to wait between "periodic" fsyncs
+    *Default Value:* 10000
+
+*Default Value:* batch
+
+*** NOTE: In the event of an unexpected shutdown, Cassandra can lose up to the sync period
or more if the sync is delayed. If using "batch" mode, it is recommended to store commitlogs
in a separate, dedicated device.**
+
+
+- ``commitlog_directory``: This option is commented out by default When running on magnetic
HDD, this should be a separate spindle than the data directories. If not set, the default
directory is $CASSANDRA_HOME/data/commitlog.
+
+*Default Value:* /var/lib/cassandra/commitlog
+
+- ``commitlog_compression``: Compression to apply to the commitlog. If omitted, the commit
log will be written uncompressed.  LZ4, Snappy, and Deflate compressors are supported.
+
+(Default Value: (complex option)::
+
+    #   - class_name: LZ4Compressor
+    #     parameters:
+    #         -
+
+- ``commitlog_total_space_in_mb``: Total space to use for commit logs on disk.
+
+If space gets above this value, Cassandra will flush every dirty CF in the oldest segment
and remove it. So a small total commitlog space will tend to cause more flush activity on
less-active columnfamilies.
+
+The default value is the smaller of 8192, and 1/4 of the total space of the commitlog volume.
+
+*Default Value:* 8192
 
 .. _memtables:
 


Mime
View raw message