hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3979) Fix hsync and hflush semantics.
Date Sat, 03 Nov 2012 22:25:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13490113#comment-13490113
] 

Lars Hofhansl commented on HDFS-3979:
-------------------------------------

Hi Kan, the only difference between v2 and v3 is that in v3 the "fsync" metric is updated
after the actual sync to the FS (BlockReceiver.flushOrSync).

This exposes the race condition we want to fix and makes TestHSync fail almost every run (the
client return from hsync before the datanode could update the metric). With the rest of this
patch applies this race is removed and TestHSync never fails.

So now we have a test case for the race condition.

[~vicaya] The existing tests: TestFiPipelines and TestFiHFlush do not cover the other scenarios
you worry about?

                
> Fix hsync and hflush semantics.
> -------------------------------
>
>                 Key: HDFS-3979
>                 URL: https://issues.apache.org/jira/browse/HDFS-3979
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node, hdfs client
>    Affects Versions: 0.22.0, 0.23.0, 2.0.0-alpha
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>         Attachments: hdfs-3979-sketch.txt, hdfs-3979-v2.txt, hdfs-3979-v3.txt
>
>
> See discussion in HDFS-744. The actual sync/flush operation in BlockReceiver is not on
a synchronous path from the DFSClient, hence it is possible that a DN loses data that it has
already acknowledged as persisted to a client.
> Edit: Spelling.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message