hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ma Hongxu <inte...@outlook.com>
Subject Re: sync related issues in hdfs IO
Date Mon, 12 Dec 2016 10:11:03 GMT
Hi paul

Question 2, I agreed with you.

Or the comment means: "since flush do not promise data has been written,  so we need sync
here to promise it..."


Overall it do sync rather than flush here.


在 12/12/2016 15:56, Paul Guo 写道:

1) In MirroredAppendOnly_FlushAndClose()

    {

        int     ret;


        errno = 0;


        ret = FileSync(open->primaryFile);

        if (ret != 0)

            *primaryError = errno;

    }


    FileClose(open->primaryFile);



If I understand the code correctly, MirroredAppendOnly_DoOpen() sets

O_SYNC in file flags which leads to a hsync in FileClose(). So it looks

that FileSync() is not needed here to protect the data integrity.


2)

 /*

 * flush hdfs file

 *

 * NB: hdfs flush do NOT promise that data has been written on disk

 * after flush, data can be read by others

 */

 int

 HdfsFileSync(File file)

It seems that HdfsFileSync finally does hsync finally, so I assume this
comment is wrong, right?



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message