hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6240) Rename operation is not consistent between different implementations of FileSystem
Date Wed, 16 Sep 2009 18:59:57 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756176#action_12756176
] 

Sanjay Radia commented on HADOOP-6240:
--------------------------------------

Sanjay> atomic if fs supports it and is within the same partition
Doug> And if an FS does not implement it atomically, should it be an error, or just not
atomic? If not atomic, that may create some confusion, no?
Turns out locaFS is implemented using Java io and hence it does not guarantee the atomicity.
We haven't offered such atomicity in the past (e.g. mkdirs, create, or rename) consistently
across all FSs and I don't think we will
be able to in the future.

Doug> Then perhaps we should encourage all applications to use symlinks as the fundamental,
reliable, atomic overwriting rename?
*But even symlinks or the atomic symlink replacement will not be offered consistently across
all FSs.*

Hence I believe we should be guided by what spec we can and want to support on hdfs.

The issue that Konstantine as raised is that if rename is worded too strictly then it prevents
someone from building the super-duper dynamically positionable NN in the future. 

Many apps don't care about atomicity of rename.
For those that do we can provide an API: FileContext#isWithinSamePartition(path1, path2).
(Owen's idea).

*Summary:* leave the wording loose as I have suggested ("atomic if fs supports it and is within
the same partition")
and add the method to check in the same partition.



> Rename operation is not consistent between different implementations of FileSystem
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-6240
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6240
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>             Fix For: 0.21.0
>
>
> The rename operation has many scenarios that are not consistently implemented across
file systems.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message