cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
Date Thu, 07 Nov 2013 19:13:19 GMT


Benedict commented on CASSANDRA-3578:

bq. when we see a corrupted columns (which are because the os or the force didn't complete
the fsync completely).

Or because we haven't flushed because we didn't know there were unfinished writes behind us
- this is easily solved with a dirty counter, but see above discussion for other issues with

bq. We can do that with another counter which holds the bytes written by all the threads and
comparing it with the allocated..... We dont need lock in that case.

Counting bytes written isn't safe, as it is unordered - so we could have A allocate, B begin
sync, C allocate, C write, B see counters equal, sync, A update counter. B thought it had
got A, but it hadn't. Unless we spin in the allocator until a double-check of both counters
is the same, but this is an unpleasant solution. The lock is asynchronous from the POV of
the writers, only the sync() ever blocks, which is better than it spinning for an undetermined
amount of time.

> Multithreaded commitlog
> -----------------------
>                 Key: CASSANDRA-3578
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jonathan Ellis
>            Assignee: Vijay
>            Priority: Minor
>              Labels: performance
>         Attachments: 0001-CASSANDRA-3578.patch,, Current-CL.png,
Multi-Threded-CL.png, parallel_commit_log_2.patch
> Brian Aker pointed out a while ago that allowing multiple threads to modify the commitlog
simultaneously (reserving space for each with a CAS first, the way we do in the SlabAllocator.Region.allocate)
can improve performance, since you're not bottlenecking on a single thread to do all the copying
and CRC computation.
> Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes doable.
> (moved from CASSANDRA-622, which was getting a bit muddled.)

This message was sent by Atlassian JIRA

View raw message