kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jay Kreps (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-191) Investigate removing the synchronization in Log.flush
Date Sun, 06 Nov 2011 15:44:51 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145022#comment-13145022
] 

Jay Kreps commented on KAFKA-191:
---------------------------------

Looks like fsync does block writes, which is kind of weak. This thread is really helpful:
  http://antirez.com/post/fsync-different-thread-useless.html

The comments do mention that fsyncdata() (aka FileChannel.force(false)) somehow doesn't interfere,
which doesn't make sense to me, as for use since we always append, FileChannel.force should
do the same thing regardless of whether we force metadata or not because we are changing the
file size which must be flushed. 
                
> Investigate removing the synchronization in Log.flush
> -----------------------------------------------------
>
>                 Key: KAFKA-191
>                 URL: https://issues.apache.org/jira/browse/KAFKA-191
>             Project: Kafka
>          Issue Type: New Feature
>          Components: core
>    Affects Versions: 0.8
>            Reporter: Jay Kreps
>            Assignee: Neha Narkhede
>         Attachments: KAFKA-191.patch
>
>
> Currently we have the following synchronization in Log.scala:
> 1. append, roll, and flush all share a write lock
> 2. read is non-blocking
> Unfortunately this means that flush time latency is added to appends (even if the flush
is done by a background thread). To fix this we should investigate a scheme to make append
and flush not block each other.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message