hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-744) Support hsync in HDFS
Date Sat, 19 May 2012 00:58:07 GMT

    [ https://issues.apache.org/jira/browse/HDFS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279378#comment-13279378
] 

Todd Lipcon commented on HDFS-744:
----------------------------------

Here's my opinion: let's create a {{syncFs(SyncType type)}} and an {{enum SyncType { HFLUSH,
HSYNC }; }}. Then, should we add a new type of sync (eg {{HSYNC_ON_ONLY_ONE_NODE}}) we can
do it compatibly without an explosion of APIs.

bq. Re: sending the flag along with the last packet. I was seeing this more as a set of flags
the describe the stream to the DN (like a file descriptor). For this specific flag it would
be ok to only send it with the last packet of a block, for other (future) flags that might
not be the case. Do you feel strongly about this? Happy to change it, if you do

If the flags are with the packet, then I think they should explicitly describe what to do
with that packet -- or cause a specific action at the time the packet is received. The strange
thing about SYNC_BLOCK is that it's not a command, but rather an indicator of some action
to do in the future. So I think it makes sense to either attach it to the empty end-of-block
(indicating the action to take at that point), or make it implicit that if you ever hsync()
at any point in a block, then it will also hsync() the close of the block. Thoughts?
                
> Support hsync in HDFS
> ---------------------
>
>                 Key: HDFS-744
>                 URL: https://issues.apache.org/jira/browse/HDFS-744
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: data-node, hdfs client
>            Reporter: Hairong Kuang
>            Assignee: Lars Hofhansl
>         Attachments: HDFS-744-trunk-v2.patch, HDFS-744-trunk-v3.patch, HDFS-744-trunk-v4.patch,
HDFS-744-trunk-v5.patch, HDFS-744-trunk.patch, hdfs-744-v2.txt, hdfs-744-v3.txt, hdfs-744.txt
>
>
> HDFS-731 implements hsync by default as hflush. As descriibed in HADOOP-6313, the real
expected semantics should be "flushes out to all replicas and all replicas have done posix
fsync equivalent - ie the OS has flushed it to the disk device (but the disk may have it in
its cache)." This jira aims to implement the expected behaviour.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message