hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lium...@apache.org
Subject hadoop git commit: HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran
Date Tue, 03 Jan 2017 21:10:15 GMT
Repository: hadoop
Updated Branches:
  refs/heads/branch-2.8 7392a5a72 -> d61af93be

HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores.
Contributed by Steve Loughran

(cherry picked from commit 451efb08fe0680d002c6856c104ebb366acee8a0)

Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/d61af93b
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/d61af93b
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/d61af93b

Branch: refs/heads/branch-2.8
Commit: d61af93befc1180badf5fab406ff94deb6b1f3c4
Parents: 7392a5a
Author: Mingliang Liu <liuml07@apache.org>
Authored: Tue Jan 3 13:08:38 2017 -0800
Committer: Mingliang Liu <liuml07@apache.org>
Committed: Tue Jan 3 13:09:53 2017 -0800

 .../site/markdown/filesystem/introduction.md    | 33 ++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/filesystem/introduction.md
index f6db557..2e6f19b 100644
--- a/hadoop-common-project/hadoop-common/src/site/markdown/filesystem/introduction.md
+++ b/hadoop-common-project/hadoop-common/src/site/markdown/filesystem/introduction.md
@@ -392,3 +392,36 @@ Object stores with these characteristics, can not be used as a direct
 for HDFS. In terms of this specification, their implementations of the
 specified operations do not match those required. They are considered supported
 by the Hadoop development community, but not to the same extent as HDFS.
+#### Timestamps
+The HDFS filesystem does not update the modification time while it is being written to.
+* `FileSystem.create()` creation: a zero-byte file is listed; creation and modification time
+  set to the current time as seen on the NameNode.
+* Writes to a file via the output stream returned in the `create()` call: the modification
+  time *does not change*.
+* When `OutputStream.close()` is called, all remaining data is written, the file closed and
+  the NameNode updated with the final size of the file. The modification time is set to
+  the time the file was closed.
+* Opening a file for appends via an `append()` operation does not change the modification
+  time of the file until the `close()` call is made on the output stream.
+* `FileSystem.setTimes()` can be used to explicitly set the time on a file.
+* The rarely used operations:  `FileSystem.concat()`, `createSnapshot()`, `createSymlink()`
+  `truncate()` all update the modification time.
+Other filesystems may have different behaviors.
+Object stores have a significantly simpler view of time:
+ * The file only becomes visible at the end of the write operation; this also sets
+   the creation time of the file.
+ * The timestamp is likely to be in UTC or the TZ of the object store. If the
+   client is in a different timezone, the timestamp may be ahead or behind that
+   of the client.
+ * A file's modification time is always the same as its creation time.
+ * The `FileSystem.setTimes()` operation to set file timestamps will generally be ignored.
+ * If `FileSystem.append()` is supported, the changes and modification time
+ are likely to only become visible after the output stream is closed.

To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org

View raw message