cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-10202) simplify CommitLogSegmentManager
Date Mon, 20 Jun 2016 20:04:57 GMT


Benedict commented on CASSANDRA-10202:

For the record - the alternative may appear more difficult to follow at first blush, but I
can assure you the prior implementation was no less custom - and in its history in fact had
many subtle bugs that were missed despite its apparent ease to follow.  The difference is
the new version named it what it was and abstracted it, and by gaining this it becomes a target
for criticism.

The prior approach has multiple lists that are effectively chained together, clumsily, into
a new composite linked list, which concurrency-wise is much more dangerous. Managing atomicity
between these lists is difficult, as they are not designed to know of each other. These transitions
were buggy at various times.  It has no stress tests besides the commitlog stress test.

On the safety of the concurrent algorithm for maintaining this logical linked list, I'm pretty
certain the new algorithm was easier to verify.  I've written a *lot* of variants of concurrent
linked lists. This one kept concurrency tightly to an absolute minimum - much less than the
prior one - by only permitting very few operations to access it in a lock-free manner.  The
remainder used mutual exclusion.  The nice bit here is that those instances of mutual exclusion
were not on the critical path (but *are* in the prior version - which is possibly a meaningful
difference, especially for mid-way thread-per-core, where stalling threads might cause more
of a latency spike).

So on that front I think your arguments are pretty fundamentally flawed.  

A reasonable rejection might be that we have a "battle tested custom linked list" already
so why replace it with another? But we have a lot of battle tested things that are very broken,
so I don't think that works as an argument either.  Our battles don't appear to have high
information value.

However, it did not simplify the rest of the CommitLog very much at all, and so on that front
it was a failure.  Possibly it made the rest worse, I can't recall. I was on the fence on
the changes when I made them, so I have no problem with you dismissing them, but I do have
an issue with you doing it for entirely the wrong reasons.

I think it is a real shame the project is still ideologically opposed to custom algorithms.
 A core piece of infrastructure with weird characteristics like Cassandra needs them.  Whether
or not we need this one, I have no strong opinion, but this seems like a blindly dogmatic
or emotive rejection rather than a well considered one.

> simplify CommitLogSegmentManager
> --------------------------------
>                 Key: CASSANDRA-10202
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Local Write-Read Paths
>            Reporter: Jonathan Ellis
>            Assignee: Branimir Lambov
>            Priority: Minor
> Now that we only keep one active segment around we can simplify this from the old recycling

This message was sent by Atlassian JIRA

View raw message