hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Bota (Jira)" <j...@apache.org>
Subject [jira] [Work started] (HADOOP-16484) S3A to warn or fail if S3Guard is disabled
Date Mon, 28 Oct 2019 10:34:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Work on HADOOP-16484 started by Gabor Bota.
> S3A to warn or fail if S3Guard is disabled
> ------------------------------------------
>                 Key: HADOOP-16484
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16484
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.2.0
>            Reporter: Steve Loughran
>            Assignee: Gabor Bota
>            Priority: Minor
> A seemingly recurrent problem with s3guard is "people who think S3Guard is turned on
but really it isn't"
> It's not immediately obvious this is the case, and the fact S3Guard is off tends to surface
after some intermittent failure has actually been detected.
> Propose: add a configuration parameter which chooses what to do when an S3A FS is instantiated
without S3Guard
> * silent : today; do nothing.
> * status: give s3guard on/off status
> * inform: log FS is instantiated without s3guard
> * warn: Warn that data may be at risk in workflows
> * fail
> deployments could then choose which level of reaction they want. I'd make the default
"inform" for now; any on-prem object store deployment should switch to silent, and if you
really want strictness, fail is the ultimate option

This message was sent by Atlassian Jira

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message