hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Mackrory (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13760) S3Guard: add delete tracking
Date Thu, 20 Apr 2017 21:59:04 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Mackrory updated HADOOP-13760:
-----------------------------------
    Attachment: HADOOP-13760-HADOOP-13345.001.patch

Attaching an initial patch to start getting some review on the basic mechanisms here. I've
tested this with 'mvn verify', and with '-Ds3guard -Dlocal' and '-Ds3guard -Ddynamo'. I've
added a couple of new tests that specifically test things that do fail due to S3 inconsistency,
but I still need to add a couple more. There's also a switch / case where there is intentional
fall-through that I need ignore in findbugs if nobody has objections to that.

I still intend to add a couple more test cases (especially covering multiple cycles of renaming
or deleting-and-creating using the same set of names, and empty directories) and I'm sure
there'll be a couple more cases I find where this isn't entirely robust (and there's a TODO
I should resolve), but I think this is complete enough to start doing some code reviews.

> S3Guard: add delete tracking
> ----------------------------
>
>                 Key: HADOOP-13760
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13760
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Aaron Fabbri
>            Assignee: Sean Mackrory
>         Attachments: HADOOP-13760-HADOOP-13345.001.patch
>
>
> Following the S3AFileSystem integration patch in HADOOP-13651, we need to add delete
tracking.
> Current behavior on delete is to remove the metadata from the MetadataStore.  To make
deletes consistent, we need to add a {{isDeleted}} flag to {{PathMetadata}} and check it when
returning results from functions like {{getFileStatus()}} and {{listStatus()}}.  In HADOOP-13651,
I added TODO comments in most of the places these new conditions are needed.  The work does
not look too bad.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message