hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitri Chmelev (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16090) deleteUnnecessaryFakeDirectories() creates unnecessary delete markers in a versioned S3 bucket
Date Tue, 05 Feb 2019 18:31:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16761089#comment-16761089
] 

Dmitri Chmelev commented on HADOOP-16090:
-----------------------------------------

{quote}So do it when createFile() is kicked off? Possibly, though as the PUT doesn't manifest
data until the write completes, that's not ideal. But I'm thinking about what we could maybe
do there, especially given we do have a pool of threads to play with.
{quote}
Not just createFile(). I was thinking of adding a pre-check in the paths that can end up creating
objects in S3 (create, close, mkdir, etc.) to call getFileStatus() on the parent directory
before initiating the rest of the operation. If the result is either a fake dir or nothing
is found, then we know that we are "likely" about to create the first dirent if the operation
succeeds. We can then cache the hint as part of the operation context and conditionally invoke
the cleanup callback if the hint is present. Note that false positives are fine here from
the point of view of correctness, because they will lead to multiple attempts to cleanup.

 

> deleteUnnecessaryFakeDirectories() creates unnecessary delete markers in a versioned
S3 bucket
> ----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-16090
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16090
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.1
>            Reporter: Dmitri Chmelev
>            Priority: Minor
>
> The fix to avoid calls to getFileStatus() for each path component in deleteUnnecessaryFakeDirectories()
(HADOOP-13164) results in accumulation of delete markers in versioned S3 buckets. The above
patch replaced getFileStatus() checks with a single batch delete request formed by generating
all ancestor keys formed from a given path. Since the delete request is not checking for existence
of fake directories, it will create a delete marker for every path component that did not
exist (or was previously deleted). Note that issuing a DELETE request without specifying
a version ID will always create a new delete marker, even if one already exists ([AWS S3
Developer Guide|https://docs.aws.amazon.com/AmazonS3/latest/dev/RemDelMarker.html])
> Since deleteUnnecessaryFakeDirectories() is called as a callback on successful writes
and on renames, delete markers accumulate rather quickly and their rate of accumulation is
inversely proportional to the depth of the path. In other words, directories closer to the
root will have more delete markers than the leaves.
> This behavior negatively impacts performance of getFileStatus() operation when it has
to issue listObjects() request (especially v1) as the delete markers have to be examined when
the request searches for first current non-deleted version of an object following a given
prefix.
> I did a quick comparison against 3.x and the issue is still present: [https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2947|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L2947]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message