hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-3979) Fix hsync and hflush semantics.
Date Sat, 13 Oct 2012 05:15:03 GMT

     [ https://issues.apache.org/jira/browse/HDFS-3979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HDFS-3979:
--------------------------------

    Attachment: hdfs-3979-v3.txt

This little change makes TestHSync fail most of the time - without the rest of the patch,
and never with this patch.

(In HDFS-744 I had avoided this race, by updating the sync metric first. I know that was a
hack... By updating the metric last in BlockReceiver.flushOrSync, this race becomes apparent
again).

We do have pipeline tests that seem to verify correct pipeline behavior in the face of failures
via fault injection: TestFiPipelines and TestFiHFlush.

In terms of the API3/API4 discussion, I think we agree that hflush should follow API4, right?
(otherwise we'd have unduly complex code)

                
> Fix hsync and hflush semantics.
> -------------------------------
>
>                 Key: HDFS-3979
>                 URL: https://issues.apache.org/jira/browse/HDFS-3979
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node, hdfs client
>    Affects Versions: 0.22.0, 0.23.0, 2.0.0-alpha
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>         Attachments: hdfs-3979-sketch.txt, hdfs-3979-v2.txt, hdfs-3979-v3.txt
>
>
> See discussion in HDFS-744. The actual sync/flush operation in BlockReceiver is not on
a synchronous path from the DFSClient, hence it is possible that a DN loses data that it has
already acknowledged as persisted to a client.
> Edit: Spelling.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message