hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ma Hongxu <inte...@outlook.com>
Subject Re: sync related issues in hdfs IO
Date Mon, 12 Dec 2016 10:11:03 GMT
Hi paul

Question 2, I agreed with you.

Or the comment means: "since flush do not promise data has been written,  so we need sync
here to promise it..."

Overall it do sync rather than flush here.

在 12/12/2016 15:56, Paul Guo 写道:

1) In MirroredAppendOnly_FlushAndClose()


        int     ret;

        errno = 0;

        ret = FileSync(open->primaryFile);

        if (ret != 0)

            *primaryError = errno;



If I understand the code correctly, MirroredAppendOnly_DoOpen() sets

O_SYNC in file flags which leads to a hsync in FileClose(). So it looks

that FileSync() is not needed here to protect the data integrity.



 * flush hdfs file


 * NB: hdfs flush do NOT promise that data has been written on disk

 * after flush, data can be read by others



 HdfsFileSync(File file)

It seems that HdfsFileSync finally does hsync finally, so I assume this
comment is wrong, right?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message