cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6809) Compressed Commit Log
Date Fri, 13 Mar 2015 19:30:42 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14360980#comment-14360980
] 

Benedict commented on CASSANDRA-6809:
-------------------------------------

The idea of making it a "hard limit" instead of the concrete number is to fix it as something
much larger than you would like it to be, but no larger than you really must expect it to
go, so that it can scale gracefully with some blips and avoid those blips having severe negative
repurcussions. My expectation is that whatever strategy we use here we will also use for non-compressed
once we migrate to manually managed memory buffers, and it seems to me always having deallocations
trail utilisation by 1 (so the first to become unused we do not deallocate, but if another
becomes unused we release one of the two)

> Compressed Commit Log
> ---------------------
>
>                 Key: CASSANDRA-6809
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benedict
>            Assignee: Branimir Lambov
>            Priority: Minor
>              Labels: docs-impacting, performance
>             Fix For: 3.0
>
>         Attachments: ComitLogStress.java, logtest.txt
>
>
> It seems an unnecessary oversight that we don't compress the commit log. Doing so should
improve throughput, but some care will need to be taken to ensure we use as much of a segment
as possible. I propose decoupling the writing of the records from the segments. Basically
write into a (queue of) DirectByteBuffer, and have the sync thread compress, say, ~64K chunks
every X MB written to the CL (where X is ordinarily CLS size), and then pack as many of the
compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message