hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-744) Support hsync in HDFS
Date Mon, 07 May 2012 15:38:50 GMT

    [ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269701#comment-13269701

stack commented on HDFS-744:

Patch looks good to me.

Were you going to expose a file mode instead of a boolean 'sync' flag?   Sync might be a nebulous
notion at this stage in hdfs's evolution but I suppose if only one thing to set on the exposed
mode, its best keep it basic for now (Rather than sync, call it fsync or 'force', the name
of the method you call on java file channel?)

The sync flag in the checksum filesystem seems to line up w/ the create parent flag in the
method call.  Thats probably not what you want?

bq. Turns out no new DataTransferProtocol option is needed, the client can send a sync packet
to indicate that the current block needs to be sync'ed even before it is closed.

Does this mean we don't need to set the sync flag for the whole filesystem such that on close
we call filechannel force?  We can do it on a file-by-file basis?

Are you thinking we'd work on optimizations later, after this patch went in; e.g. sync only
the first block in pipeline while we are replicating to the other pipeline members, etc.?

> Support hsync in HDFS
> ---------------------
>                 Key: HDFS-744
>                 URL: https://issues.apache.org/jira/browse/HDFS-744
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Hairong Kuang
>         Attachments: hdfs-744.txt
> HDFS-731 implements hsync by default as hflush. As descriibed in HADOOP-6313, the real
expected semantics should be "flushes out to all replicas and all replicas have done posix
fsync equivalent - ie the OS has flushed it to the disk device (but the disk may have it in
its cache)." This jira aims to implement the expected behaviour.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message