hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-15575) Add a way for an FS instance to say "really, no trash interval at all"
Date Mon, 02 Jul 2018 10:52:00 GMT
Steve Loughran created HADOOP-15575:

             Summary: Add a way for an FS instance to say "really, no trash interval at all"
                 Key: HADOOP-15575
                 URL: https://issues.apache.org/jira/browse/HADOOP-15575
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs, fs/s3
    Affects Versions: 3.1.0
            Reporter: Steve Loughran

Object stores like S3 often offer [object versioning|https://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectVersioning.html]
as a way to recover from accidental deletions. There is no need to use the Fs Shell trash

Proposed: a way for an FS instance to tell the shell to not use trash, even if the client
has configured it. The current {{getServerDefaults().getTrashInterval()}} returns a value
from the server, but it *only* overrides the ""fs.trash.interval" if it !=0. So if a server
says "don't use trash" it gets ignored.

If a special value (-1) is returned (or we use a new field?), FS instances can return to the
shell saying "no need for trash". Then you can turn it on for a bucket by bucket basis; any
store with the feature enabled can do the same thing.

Alternative option: A special S3A-aware trash policy which does a getFileSystem.getConf.getBoolean("fs.s3a.trash.skip")
& skips trash if set. This will all you to disable it on a bucket-by-bucket basis. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message