cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Brown (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-6809) Compressed Commit Log
Date Wed, 20 Aug 2014 05:15:20 GMT


Jason Brown commented on CASSANDRA-6809:

bq. It's also potentially less capable of utilising machines with... / won't utilise all of
our users' hardware maximally

Let's face it, if you're hitting the CL hard, you're also hitting the memtables, et al, pretty
hard, as well, due a consistently large number of writes coming in. As long as the CL doesn't
fall behind/take up too many resources/does it's job and doesn't complain, I think that's
the sweet spot. I'd rather have the resources optimized for the client-facing path.

bq. dropping CLS recycling ... never really convinced it was worth the effort

I tend to agree with this. I'd like to think it's a nice win, but I have no numbers to back
it up. I think the pre-allocation is good (and a cheap operation, at least by LOC), but the
recycling I'm not sure of. [~rbranson], it's been a few years since you implemented this in
CASSANDRA-3411, how do feel about it now, with several years of production experience?

bq. We'll want to limit the concurrency in this case

Does this statement apply to this ticket or CASSANDRA-7075? Seems more apropos to the later.

> Compressed Commit Log
> ---------------------
>                 Key: CASSANDRA-6809
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benedict
>            Assignee: Branimir Lambov
>            Priority: Minor
>              Labels: performance
>             Fix For: 3.0
> It seems an unnecessary oversight that we don't compress the commit log. Doing so should
improve throughput, but some care will need to be taken to ensure we use as much of a segment
as possible. I propose decoupling the writing of the records from the segments. Basically
write into a (queue of) DirectByteBuffer, and have the sync thread compress, say, ~64K chunks
every X MB written to the CL (where X is ordinarily CLS size), and then pack as many of the
compressed chunks into a CLS as possible.

This message was sent by Atlassian JIRA

View raw message