hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andras Bokor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15361) RawLocalFileSystem should use Java nio framework for rename
Date Tue, 10 Apr 2018 17:28:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432622#comment-16432622
] 

Andras Bokor commented on HADOOP-15361:
---------------------------------------

Patch 02 keeps the behavior of the old logic where necessary.

Let's see what does Hadoop QA think.

> RawLocalFileSystem should use Java nio framework for rename
> -----------------------------------------------------------
>
>                 Key: HADOOP-15361
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15361
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Andras Bokor
>            Assignee: Andras Bokor
>            Priority: Major
>              Labels: incompatibleChange
>         Attachments: HADOOP-15361.01.patch, HADOOP-15361.02.patch
>
>
> Currently RawLocalFileSystem uses a fallback logic for cross-volume renames. The fallback
logic is a copy-on-fail logic so when rename fails it copies the source then delete it.
>  An additional fallback logic was needed for Windows to provide POSIX rename behavior.
> Due to the fallback logic RawLocalFileSystem does not pass the contract tests (HADOOP-13082).
> With using Java nio framework both could be eliminated since it is not platform dependent
and provides cross-volume rename.
> In addition the fallback logic for Windows is not correct since Java io overrides the
destination only if the source is also a directory but handleEmptyDstDirectoryOnWindows method
checks only the destination. That means rename allows to override a directory with a file
on Windows but not on Unix.
> File#renameTo and Files#move are not 100% compatible:
>  If the source is a directory and the destination is an empty directory File#renameTo
overrides the source but Files#move is does not. We have to use {{StandardCopyOption.REPLACE_EXISTING}}
but it overrides the destination even if the source or the destination is a file. So to make
them compatible we have to check that the either the source or the destination is a directory
before we add the copy option.
> I think the correct strategy is
>  * Where the contract test passed so far it should pass after this
>  * Where the contract test failed because of Java specific think and not because of the
fallback logic we should keep the original behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message