hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jagdish Kewat (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12837) FileStatus.getModificationTime not working on S3
Date Sat, 27 Feb 2016 12:45:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170575#comment-15170575

Jagdish Kewat commented on HADOOP-12837:

Well, the above mentioned workaround of the _SUCCESS file works in my case since the content
of the directory in question isn't expected to change after it is created. However in case
of frequently updating directory contents that won't work. 
For that case one needs to dig deeper in the directory / sub-directories and determine mtime
of each file and finally return the max value as mtime of the directory, however, that would
be an expensive operation, particularly in case of huge directories / subdirectories.

For now the workaround seems to be working for me. You may want to keep this ticket in backlog
if this happens to find priority. Feel free to close if otherwise.

Thanks guys for sharing your inputs.


> FileStatus.getModificationTime not working on S3
> ------------------------------------------------
>                 Key: HADOOP-12837
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12837
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>            Reporter: Jagdish Kewat
> Hi Team,
> We have observed an issue with the FileStatus.getModificationTime() API on S3 filesystem.
The method always returns 0.
> I googled for this however couldn't find any solution as such which would fit in my scheme
of things. S3FileStatus seems to be an option however I would be using this API on HDFS as
well as S3 both so can't go for it.
> I tried to run the job on:
> * Release label:emr-4.2.0
> * Hadoop distribution:Amazon 2.6.0
> * Hadoop Common jar: hadoop-common-2.6.0.jar
> Please advise if any patch or fix available for this.
> Thanks,
> Jagdish

This message was sent by Atlassian JIRA

View raw message