cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ariel Weisberg (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-8771) Remove commit log segment recycling
Date Wed, 11 Feb 2015 18:50:12 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316754#comment-14316754
] 

Ariel Weisberg edited comment on CASSANDRA-8771 at 2/11/15 6:49 PM:
--------------------------------------------------------------------

There is also the option of calling fallocate via JNA. What that does is an open question
for me as it is file system specific. Red Hat in their performance guide claim that it will
encourage better allocation.

With compression, at least the current proposed implementation it is a little undesirable
because it doesn't get very close to the target size of 32 megabytes after compression shrinks
the data in the segment.


was (Author: aweisberg):
There is also the option of calling fallocate via JNA. What that does is an open question
for me as it is file system specific. Red Hat in their performance guide claim that it will
encourage better allocation.

> Remove commit log segment recycling
> -----------------------------------
>
>                 Key: CASSANDRA-8771
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8771
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Ariel Weisberg
>              Labels: commitlog
>
> For discussion
> Commit log segment recycling introduces a lot of complexity in the existing code.
> CASSANDRA-8729 is a side effect of commit log segment recycling and addressing it will
require memory management code and thread coordination for memory that the filesystem will
no longer handle for us.
> There is some discussion about what storage configurations actually benefit from preallocated
files. Fast random access devices like SSDs, or non-volatile write caches etc. make the distinction
not that great. 
> I haven't measured any difference in throughput for bulk appending vs overwriting although
it was pointed out that I didn't test with concurrent IO streams.
> What would it take to make removing commit log segment recycling acceptable? Maybe a
benchmark on a spinning disk that measures the performance impact of preallocation when there
are other IO streams?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message