hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11452) Revisit FileSystem#rename
Date Wed, 31 Dec 2014 21:43:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262447#comment-14262447
] 

Colin Patrick McCabe commented on HADOOP-11452:
-----------------------------------------------

In POSIX, {{rename}} is defined to be atomic.  That's not just an implementation detail, but
a fundamental guarantee.  See http://pubs.opengroup.org/onlinepubs/009695399/functions/rename.html
 The Ceph distributed filesystem supports atomic {{rename}} in the context of distributed
metadata storage.  Some proprietary systems do as well.

I would argue that if we want to create a new operation that might be similar to rename in
some ways but potentially non-atomic, we should name it something else to avoid confusion.
 Perhaps we could use our existing "move" function for this.  And if the hadoop filesystem
can't atomically rename from a to b, it should throw an exception.  This is simlar to how
POSIX rename returns EXDEV when you try to do a rename operation that wouldn't be atomic because
it spans multiple filesystems.

> Revisit FileSystem#rename
> -------------------------
>
>                 Key: HADOOP-11452
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11452
>             Project: Hadoop Common
>          Issue Type: Task
>          Components: fs
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>
> Currently in {{FileSystem}}, {{rename}} with _Rename options_ is protected and with _deprecated_
annotation. And the default implementation is not atomic.
> So this method is not able to be used outside. On the other hand, HDFS has a good and
atomic implementation. (Also an interesting thing in {{DFSClient}}, the _deprecated_ annotations
for these two methods are opposite).
> It makes sense to make public for {{rename}} with _Rename options_, since it's atomic
for rename+overwrite, also it saves RPC calls if user desires rename+overwrite.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message