cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vijay (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-3578) Multithreaded commitlog
Date Thu, 07 Nov 2013 19:07:17 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13816281#comment-13816281
] 

Vijay edited comment on CASSANDRA-3578 at 11/7/13 7:05 PM:
-----------------------------------------------------------

{quote}
You only call force when you think there is something dirty, not when the buffer does, 
{quote}
Ahaa.... that might be an over sight, we can call buffer.force all the time and let the OS
decide if it has to sync the filesystem. If we do that then we just need to stop during the
recovery/replay when we see a corrupted columns (which are because the os or the force didn't
complete the fsync completely).

{quote}
How ugly would it get to either wait for previous (in CL order) mutations before syncing
{quote}
We can do that with another counter which holds the bytes written by all the threads and comparing
it with the allocated..... We dont need lock in that case.


was (Author: vijay2win@yahoo.com):
{quote}
You only call force when you think there is something dirty, not when the buffer does, 
{quote}
Ahaa.... that might be an over sight, we can call buffer.force all the time and let the OS
decide if it has to sync the filesystem. If we do that then we just need to stop during the
recovery when we have a corrupted columns (which are because the os or the force didnt complete
the fsync completely).

{quote}
How ugly would it get to either wait for previous (in CL order) mutations before syncing
{quote}
We can do that with another counter which holds the bytes written by all the threads and comparing
it with the allocated..... We dont need lock in that case.

> Multithreaded commitlog
> -----------------------
>
>                 Key: CASSANDRA-3578
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3578
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jonathan Ellis
>            Assignee: Vijay
>            Priority: Minor
>              Labels: performance
>         Attachments: 0001-CASSANDRA-3578.patch, ComitlogStress.java, Current-CL.png,
Multi-Threded-CL.png, parallel_commit_log_2.patch
>
>
> Brian Aker pointed out a while ago that allowing multiple threads to modify the commitlog
simultaneously (reserving space for each with a CAS first, the way we do in the SlabAllocator.Region.allocate)
can improve performance, since you're not bottlenecking on a single thread to do all the copying
and CRC computation.
> Now that we use mmap'd CommitLog segments (CASSANDRA-3411) this becomes doable.
> (moved from CASSANDRA-622, which was getting a bit muddled.)



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message