cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vijay (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-3578) Multithreaded commitlog
Date Sun, 27 Oct 2013 00:17:32 GMT


Vijay commented on CASSANDRA-3578:

Hi Jonathan, 

 I must be missing where this gets persisted back to disk
First 4 bytes at the beginning of the file, may be we can get rid of it and stop when the
size and checksum doesn't match?

But the header is pretty light, and will need one additional seek every 10 seconds (it just
marks the end of the file at the beginning of the file just before fsync).

 I think allocate needs to write the length to the segment before returning
The first thing the thread does after allocation is writing the size and its checksum....
are we talking about synchronization in the allocation, so only 1 thread writes the size and
end (-1)? currently the atomic operation is only on AtomicLong (position).... 

We might be able to do something similar to the current implementation and without headers
with a Read Write lock, where write lock will ensure that we write the end (write -1 to mark
the end, lock to ensure no one else overwrites the end marker) just before fsync (but the
OS can also write before we force the buffers too)... also that might not be desirable, since
it might stall the system like the current one.

Not sure if the header is that bad though....  Let me know what you think, thanks!

> Multithreaded commitlog
> -----------------------
>                 Key: CASSANDRA-3578
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jonathan Ellis
>            Assignee: Vijay
>            Priority: Minor
>              Labels: performance
>         Attachments: 0001-CASSANDRA-3578.patch,, Current-CL.png,
Multi-Threded-CL.png, parallel_commit_log_2.patch
> Brian Aker pointed out a while ago that allowing multiple threads to modify the commitlog
simultaneously (reserving space for each with a CAS first, the way we do in the SlabAllocator.Region.allocate)
can improve performance, since you're not bottlenecking on a single thread to do all the copying
and CRC computation.
> Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes doable.
> (moved from CASSANDRA-622, which was getting a bit muddled.)

This message was sent by Atlassian JIRA

View raw message