cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ariel Weisberg (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-14062) Pluggable CommitLog
Date Tue, 05 Dec 2017 21:32:00 GMT


Ariel Weisberg commented on CASSANDRA-14062:

My concern is not so much that people use the plugin and don't upgrade when upgrading Cassandra,
but that data is lost and then we have to track down that it was an issue with interactions
between Cassandra and the plugin. The starting point for everyone is going to be that Cassandra
lost data and it's up to us to prove otherwise. With software and hardware we probably won't
have access to.

Not allowing the plugins at all isn't a viable solution so I have three routes I think we
might take.

1. If you use a commit log plugin and have issues we aren't going to get involved at all.
We assume it's the plugin. I've seen this approach taken for things like your using JNI and
getting segfaults from the JVM? Great, go create a reproducer that has no JNI or we won't
look at it.
2. The plugin is passed the current Cassandra version and won't start if it the version string
<= the supported version of the plugin and the plugins don't cheat. So we would implement
this version checking code in OSS, but the version string would be compiled into the plugin.
At least then we know the plugin and it's tests were run against the current C* version. I
would expect a commit log plugin to be well tested integration test wise, but we have no control
over it.
3. We factor this out into a proper tested interface that detects changes in contractual behavior.
This exposes much less of C* and makes explicit what we do expose. So everything in a new
package for exposing to plugins. If you are compiling a plugin it should only whitelist classes
that are in the plugin support package and generate a compile time error if anything outside
it is used.

#3 is not cheap. It could lead to code duplication or encourage us to avoid modifying code
because then we would have to duplicate it outside the plugin support package.

I am in favor of #2 for now since it's simple and puts the onus for making things work on
plugin developers. The extra work of #3 only makes sense when we want to put in the extra
effort of stabilizing the interface to make things easier for plugin developers. If we can
live with plugins being compiled and matched to specific C* releases then #2 has the same
level of safety and is easy to get up and running.

> Pluggable CommitLog
> -------------------
>                 Key: CASSANDRA-14062
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Rei Odaira
>            Assignee: Rei Odaira
>              Labels: features
>             Fix For: 4.x
>         Attachments: pluggable-commitlog-src.patch, pluggable-commitlog-test.patch
> This proposal is to make CommitLog pluggable, as discussed in [the Cassandra dev mailing
> We are developing a Cassandra plugin to store CommitLog on our low-latency Flash device
(CAPI-Flash). To do that, the original CommitLog interface must be changed to allow plugins.
Synching to CommitLog is one of the performance bottlenecks in Cassandra especially with batch
commit. I think the pluggable CommitLog will allow other interesting alternatives, such as
one using SPDK.
> Our high-level design is similar to the CacheProvider framework
> in org.apache.cassandra.cache:
> * Introduce a new interface, ICommitLog, with methods like getCurrentPosition(), add(),
shutdownBlocking(), etc.
> * CommitLog implements ICommitLog.
> * Introduce a new interface, CommitLogProvider, with a create() method, returning ICommitLog.
> * Introduce a new class FileCommitLogProvider implementing CommitLogProvider, to return
a singleton instance of CommitLog.
> * Introduce a new property in cassandra.yaml, commitlog_class_name, which specifies what
CommitLogProvider to use.  The default is FileCommitLogProvider.
> * Introduce a new class, CommitLogHelper, that loads the class specified by the commitlog_class_name
property, creates an instance, and stores it to CommitLogHelper.instance.
> * Replace all of the references to CommitLog.instance with CommitLogHelper.instance.
> Attached are two patches. "pluggable-commitlog-src.patch" is for changes in the src directory,
and "pluggable-commitlog-test.patch" is for the test directory.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message