cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-7031) Increase default commit log total space + segment size
Date Mon, 14 Apr 2014 10:27:16 GMT


Benedict commented on CASSANDRA-7031:

I ran the following simple benchmarks, with default config and with 8Gb max total space and
256Mb segment size:

\[1\]: cassandra-stress write n=10000000 -key populate=1..10000 -rate threads=50 -mode thrift
\[2\]: cassandra-stress write n=10000000 -key populate=1..1000000 -rate threads=50 -mode thrift

||measurement||default 1||new config 1||default 2||new config 2||
|latency 95th percentile| 2.2| 1.8| 2.4| 2.0|
|latency 99th percentile| 8.2| 7.9| 7.8| 8.2|
|latency 99.9th percentile| 21.1| 21.9| 62.7| 72.8|
|latency max| 386.7| 251.3| 4324.6| 3585.2|
|real op rate| 54244| 56120| 35001| 35824|
|Total operation time| 00:03:04| 00:02:58| 00:04:45| 00:04:39|

The difference to throughput is small, but measurable; the impact on latency is more pronounced,
although still small.

> Increase default commit log total space + segment size
> ------------------------------------------------------
>                 Key: CASSANDRA-7031
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Benedict
>            Assignee: Benedict
>            Priority: Trivial
>             Fix For: 2.1 beta2
> I would like to increase the default commit log total space and segment size options
for 64-bit JVMs:
> The current default of 1Gb and 32Mb is quite constrained and can have some (very minor)
negative performance implications, for no major benefit: 
> # 32Mb files are actually quite small, and if during the 10s interval we have completely
filled multiple of them (quite easy) it would be more efficient to write fewer larger files,
as we can issue fewer fsyncs and permit the OS to schedule the writes more efficiently. On
my box this has a small but noticeable impact. Although I would expect on decent server hardware
this would be smaller still, since we immediately drop the pages from cache on writing there
isn't a great deal of advantage to keeping the files so small. The only advantage I can see
is that during a drop KS/CF or other event that forces log rollover we're wasting less space
until log recycling. 128-256Mb are modest increases that seem more appropriate to me.
> # 1Gb is too small for the default total log space. We can find that we force memtable
flushes as a result of log utilisation instead of memtable occupancy quite often (esp. as
a result of increased effective memtable space from recent improvements), especially on machines
with more addressable memory. I suggest 8Gb as a minimum. The only disadvantage of having
more log data is that replay on restart may be slightly slower, but since most of the events
will be ignored it should be relatively benign, and I would rather take the penalty on startup
instead of during running, no matter how small the running penalty.

This message was sent by Atlassian JIRA

View raw message