cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ariel Weisberg (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6809) Compressed Commit Log
Date Tue, 10 Feb 2015 01:04:07 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313299#comment-14313299
] 

Ariel Weisberg commented on CASSANDRA-6809:
-------------------------------------------

* In cassandra.yaml, for commitlog_compression_threads can you document that the default is
1?
* Cap the size of the buffer pool to a fixed number of buffers. If you end up allocating more
than that number, free the memory rather then pooling.
* awaitTermination changed from block forever to block for 3600 seconds. I have never been
a fan of ExecutorService's async termination. If it was supposed to finish it's last task
it should terminate and if it doesn't then something is wrong.
* I’d still like to see the JSON field in commit log descriptor be reusable for other config
parameters.
* In CommitLogSegment constructor it is no longer truncating the file, why is it no longer
necessary?

I am more comfortable with the single threaded task generation in Benedict's diff. I would
rather see that or no multi-threading for now.

> Compressed Commit Log
> ---------------------
>
>                 Key: CASSANDRA-6809
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benedict
>            Assignee: Branimir Lambov
>            Priority: Minor
>              Labels: performance
>             Fix For: 3.0
>
>         Attachments: ComitLogStress.java, logtest.txt
>
>
> It seems an unnecessary oversight that we don't compress the commit log. Doing so should
improve throughput, but some care will need to be taken to ensure we use as much of a segment
as possible. I propose decoupling the writing of the records from the segments. Basically
write into a (queue of) DirectByteBuffer, and have the sync thread compress, say, ~64K chunks
every X MB written to the CL (where X is ordinarily CLS size), and then pack as many of the
compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message