hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Bota (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16484) S3A to warn or fail if S3Guard is disabled
Date Wed, 16 Oct 2019 15:29:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16952938#comment-16952938

Gabor Bota commented on HADOOP-16484:

I removed status (merged with inform) so at inform level it will log with 

> S3A to warn or fail if S3Guard is disabled
> ------------------------------------------
>                 Key: HADOOP-16484
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16484
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.2.0
>            Reporter: Steve Loughran
>            Assignee: Gabor Bota
>            Priority: Minor
> A seemingly recurrent problem with s3guard is "people who think S3Guard is turned on
but really it isn't"
> It's not immediately obvious this is the case, and the fact S3Guard is off tends to surface
after some intermittent failure has actually been detected.
> Propose: add a configuration parameter which chooses what to do when an S3A FS is instantiated
without S3Guard
> * silent : today; do nothing.
> * status: give s3guard on/off status
> * inform: log FS is instantiated without s3guard
> * warn: Warn that data may be at risk in workflows
> * fail
> deployments could then choose which level of reaction they want. I'd make the default
"inform" for now; any on-prem object store deployment should switch to silent, and if you
really want strictness, fail is the ultimate option

This message was sent by Atlassian Jira

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message