hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Shvachko <shv.had...@gmail.com>
Subject Re: Regarding HDFS-15567. HDFS should expose msync() API to allow downstream applications call it explicitly
Date Thu, 24 Dec 2020 21:39:51 GMT
Hi Steve,

I created HDFS-15751 <https://issues.apache.org/jira/browse/HDFS-15751> for
documenting msync API.
Would appreciate your suggestions.

Stay safe,
--Konstantin

On Mon, Dec 21, 2020 at 5:19 AM Steve Loughran <stevel@cloudera.com> wrote:

>
>
> On Fri, 18 Dec 2020 at 23:29, Konstantin Shvachko <shv.hadoop@gmail.com>
> wrote:
>
>> Hey Steve,
>>
>> Thanks for the references. I was reading but still need to understand how
>> exactly this applies to msync.
>>
>
> mainly: pull it up and it becomes part of the broader API, so needs to be
> specified in a way which can be understood by users and for implementors of
> others stores: to give their own stores the same semantics.
>
> What does the HDFS one do?
>
>
>
>> Will come up with a plan and post it on a new jira.
>> Will make sure to create it under HADOOP and ping Hadoop Common list for
>> visibility.
>>
>>
> thanks
>
>
>> You are right about ViewFS. The impl should make sure it calls msync() on
>> all mount points that enabled observer reads.
>>
>>
> That's the kind of issue this process aims to resolve. Another is to
> identify where we have HDFS-layer "quirks" and at least document them (e.g.
> how hdfs streams are thread safe, rename isn't Posix, ...) and list what we
> know breaks if you don't re-implement
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message