cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CASSANDRA-7139) Default concurrent_compactors is probably too high
Date Fri, 02 May 2014 12:24:14 GMT
Benedict created CASSANDRA-7139:
-----------------------------------

             Summary: Default concurrent_compactors is probably too high
                 Key: CASSANDRA-7139
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7139
             Project: Cassandra
          Issue Type: Improvement
          Components: Core
            Reporter: Benedict
            Priority: Minor
             Fix For: 2.0.8, 2.1 rc1


The default number of concurrent compactors is probably too high for modern hardware with
spinning disks for storage: A modern blade can easily have 24+ Cores, which would result in
a default of 24 concurrent compactions. This not only increases random IO, it also keeps around
a lot of obsoleted files for an unnecessarily long time, as each compaction keeps references
to any possibly overlapping files that it isn't itself compacting - but these can have been
obsoleted part way through by compactions that finished earlier. If you factor in the default
compaction throughput rate of 16Mb/s, anything but a single default concurrent_compactor makes
very little sense, as a single thread should always be able to handle 16Mb/s, will cause less
interference with other processes, and permits obsoleted files to be immediately removed.

See [http://imgur.com/HDqhxFp] for a graph demonstrating the result of making this change
on a box with 24-cores and 8Tb of storage (first spike is default settings)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message