hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1377) Creation time and modification time for hadoop files and directories
Date Fri, 22 Jun 2007 20:20:26 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507498

Konstantin Shvachko commented on HADOOP-1377:

- TestCreateModTime.java:  Redundant imports, method, and variable declarations:
import java.util.Collection;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.Date;
63:  private void checkFile(FileSystem fileSys, Path name, int repl)
100:    DistributedFileSystem dfs = (DistributedFileSystem) fileSys;
132:     long ctime2 = stat.getCreationTime();
133:     long mtime2 = stat.getModificationTime();

- Classes RawLocalFileStatus, InMemoryFileStatus, and S3FileStatus have almost identical implementations.
It makes sense to have one base class that provides a default FilesStatus implementation and
make those three
subclasses if the default is not good enough. The base class can be an inner class of FileSystem
or the FileStatus
itself can be declared as an abstract class instead of being an interface.

- FSConstants: The comment describing changes related to the new layout version should be
  // Current version:

- FSEditLog:
fromLogTimeStamp(UTF8) is not used anywhere

- DistributedFileSystem: Unused variable in getFileStatus()
      FileStatus stat = null;

- FSDirectory:
    long modTime = namesystem.now();
Should be accessed in a static way NameSystem.now()

> Creation time and modification time for hadoop files and directories
> --------------------------------------------------------------------
>                 Key: HADOOP-1377
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1377
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>             Fix For: 0.14.0
>         Attachments: 1377.patch, CreationModificationTime.html, CreationTime8.patch
> This issue will document the requirements, design and implementation of creation times
and modification times of hadoop files and directories.
> My proposal is to have support two additional attributes for each file and directory
in HDFS. The "creation time" is the time when the file/directory was created. It is a 8 byte
integer stored in each FSDirectory.INode. The "modification time" is the time when the last
modification occured to the file/directory. It is an 8 byte integer stored in the FSDirectory.INode.
These two fields are stored in in the FSEdits and FSImage as part of the transaction that
created the file/directory.
> My current proposal is to not support "access time" for a file/directory. It is costly
to implement and current applications might not need it.
> In the current implementation, the "modification time" for a file will be same as its
creation time because HDFS files are currently unmodifiable. Setting file attributes (e.g.
setting the replication factor) of a file does not modify the "modification time" of that
file. The "modification time" for a directory is either its creation time or the time when
the most recent file-delete or file-create occured in that directory.
> A new command named "hadoop dfs -lsl" will display the creation time and modification
time of the files/directories that it lists. The output of the existing command "hadoop dfs
-ls" will not be affected.
> The ClientProtocol will change because DFSFileInfo will have two additional fields: the
creation time and modification time of the file that it represents. This information can be
retrieved by clients thorugh the ClientProtocol.getListings() method. The FileSystem public
API will have two additional methods: getCreationTime and getModificationTime().
> The datanodes are completely transparent to this design and implementation and requires
no change.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message