hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Marquardt (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-15086) NativeAzureFileSystem file rename is not atomic
Date Thu, 21 Dec 2017 04:32:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Thomas Marquardt updated HADOOP-15086:
    Attachment: HADOOP-15086-001.patch

Attaching HADOOP-15086-001.patch.  The scope of the fix is limited to avoid regressions. 
The fix targets the bug identified by RenameReproducer.java, which has been ported to the
test case {{testMultipleRenameFileOperationsToSameDestination}}.  When multiple rename operations
are performed on the same destination file, at most one of the operations will succeed.

All tests pass against my endpoint in US West 2.

Tests run: 765, Failures: 0, Errors: 0, Skipped: 81

> NativeAzureFileSystem file rename is not atomic
> -----------------------------------------------
>                 Key: HADOOP-15086
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15086
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 2.7.3
>            Reporter: Shixiong Zhu
>            Assignee: Thomas Marquardt
>         Attachments: HADOOP-15086-001.patch, RenameReproducer.java
> When multiple threads rename files to the same target path, more than 1 threads can succeed.
It's because check and copy file in `rename` is not atomic.
> I would expect it's atomic just like HDFS.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message