hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9208) Disabling atime may fail clients like distCp
Date Wed, 07 Oct 2015 17:46:26 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947256#comment-14947256

Chris Nauroth commented on HDFS-9208:

Hello [~kihwal].

I had filed HADOOP-12390 related to this problem specifically for {{hdfs dfs -put -p}}.  I
proposed fixing it at the application layer by supporting additional flags for preserving
specific attributes.

Instead of fixing it in applications, I think I prefer your option #3 here to allow explicit
changes to atime at the NameNode layer.  I don't see any harm in allowing that.  A single
fix at the NameNode layer would help all applications that might suffer from this problem.

> Disabling atime may fail clients like distCp
> --------------------------------------------
>                 Key: HDFS-9208
>                 URL: https://issues.apache.org/jira/browse/HDFS-9208
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Kihwal Lee
>            Assignee: Mingliang Liu
> When atime is disabled, {{setTimes()}} throws an exception if the passed-in atime is
not -1.  But since atime is 0, distCp fails when it tries to set the mtime and atime. 
> There are several options:
> 1) make distCp check for 0 atime and call {{setTimes()}} with -1. I am not very enthusiastic
about it.
> 2) make NN also accept 0 atime in addition to -1, when the atime support is disabled.
> 3) support setting mtime & atime regardless of the atime support.  The main reason
why atime is disabled is to avoid edit logging/syncing during {{getBlockLocations()}} read
calls. Explicit setting can be allowed.

This message was sent by Atlassian JIRA

View raw message