hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan Kunigk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4089) SyncBehindWrites uses wrong flags on sync_file_range
Date Thu, 25 Oct 2012 22:55:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484557#comment-13484557

Jan Kunigk commented on HDFS-4089:

Hi Todd, sorry the lag in my replies. I think we can improve the naming at least.
1) I would rename the variable from SyncBehindWrites to WriteOutDirtyOSPages. Sync is misleading,
since, even though the syscall is named sync_file_range, the flags that it is used with here
do not entail sync semantics, i.e. a synchronous, i.e. blocking operation.
2) I think we should mark the appropriate section in DFSConfigKeys.java as "performance related"

This is more an issue of hygiene and, after your explanations, no longer a concern w/r to
data integrity. Thanks. 

> SyncBehindWrites uses wrong flags on sync_file_range
> ----------------------------------------------------
>                 Key: HDFS-4089
>                 URL: https://issues.apache.org/jira/browse/HDFS-4089
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node, ha
>            Reporter: Jan Kunigk
>            Priority: Minor
>         Attachments: syncBehindWrites.patch
> Hi, I stumbled upon this while trying to understand the append design recently. I am
assuming when SyncBehindWrites is enabled we do indeed want to do a complete sync after each
write. In that case the implementation seems wrong to me.
> Here's a comment from the manpage of sync_file_range on the usage of the SYNC_FILE_RANGE_WRITE
flag in solitude: "This is an asynchronous flush-to-disk operation. This is not suitable for
data integrity operations." I don't know why this syscall is invoked here instead of just

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message