hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-15183) S3Guard store becomes inconsistent after partial failure of rename
Date Fri, 19 Jan 2018 17:16:00 GMT
Steve Loughran created HADOOP-15183:

             Summary: S3Guard store becomes inconsistent after partial failure of rename
                 Key: HADOOP-15183
                 URL: https://issues.apache.org/jira/browse/HADOOP-15183
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
    Affects Versions: 3.0.0
            Reporter: Steve Loughran

If an S3A rename() operation fails partway through, such as when the user doesn't have permissions
to delete the source files after copying to the destination, then the s3guard view of the
world ends up inconsistent. In particular the sequence

 (assuming src/file* is a list of files file1...file10 and read only to caller)
# create file dest/file1
# delete file dest/file1
# rename src/file* dest/ 

You will not see file1 in the listing, because it will have a tombstone marker and the update
at the end of the rename() didn't take place: the old data is still there.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

View raw message