hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Demoor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13946) Document how HDFS updates timestamps in the FS spec; compare with object stores
Date Tue, 03 Jan 2017 22:37:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15796426#comment-15796426
] 

Thomas Demoor commented on HADOOP-13946:
----------------------------------------

Another late review remark:
bq. The file only becomes visible at the end of the write operation; this also sets the creation
time of the file.

On S3 an object (file) is only visible at the end of the write operation, but the creation
/ modification timestamp (Last-Modified HTTP header) is that of the *start* of the write operation.


> Document how HDFS updates timestamps in the FS spec; compare with object stores
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-13946
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13946
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: documentation, fs
>    Affects Versions: 2.7.3
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>             Fix For: 2.8.0, 3.0.0-alpha2
>
>         Attachments: HADOOP-13946-001.patch
>
>
> SPARK-17159 shows that the behavior of when HDFS updates timestamps isn't well documented.
Document these in the FS spec.
> I'm not going to add tests for this, as it is so very dependent on FS implementations,
as in "POSIX filesystems may behave differently from HDFS". If someone knows what happens
there, their contribution is welcome.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message