cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6809) Compressed Commit Log
Date Fri, 23 Jan 2015 01:04:07 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288522#comment-14288522
] 

Benedict commented on CASSANDRA-6809:
-------------------------------------

It's a bit late at night to throw this together, so this is only an expression of intent,
but here's quick hour's attempt at the approach I am advocating. It should be complete, but
ignoring the extension ideas or exploring any other avenues, and with only around 97% care
(i.e. I'm disclaiming minor mistakes). All I've done is split the sync() into a prepareSync()
which returns a List of Callable<Runnable> - the callable is the compression and write-to-buffer
work, and the Runnable is the FS barrier work. The former is submitted to an executor service,
which may be an inline executor. The result is then executed inline in the sync thread.

I think it is a pretty simple change, and it is very clear what its behaviour is. Let me know
what you think. I think we should probably split this discussion into its own ticket, though,
since the work you've done on compression is good to commit as is IMO.

Branch [here|https://github.com/belliottsmith/cassandra/commits/6809-extend]

> Compressed Commit Log
> ---------------------
>
>                 Key: CASSANDRA-6809
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benedict
>            Assignee: Branimir Lambov
>            Priority: Minor
>              Labels: performance
>             Fix For: 3.0
>
>         Attachments: ComitLogStress.java, logtest.txt
>
>
> It seems an unnecessary oversight that we don't compress the commit log. Doing so should
improve throughput, but some care will need to be taken to ensure we use as much of a segment
as possible. I propose decoupling the writing of the records from the segments. Basically
write into a (queue of) DirectByteBuffer, and have the sync thread compress, say, ~64K chunks
every X MB written to the CL (where X is ordinarily CLS size), and then pack as many of the
compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message