hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HADOOP-14645) add a predicate/option to probe an S3A FS for being consistent
Date Fri, 04 Aug 2017 14:10:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran resolved HADOOP-14645.
    Resolution: Won't Fix

> add a predicate/option to probe an S3A FS for being consistent
> --------------------------------------------------------------
>                 Key: HADOOP-14645
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14645
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: HADOOP-13345
>            Reporter: Steve Loughran
> The committer work shows It's useful for s3a-aware code to be able to ask if the FS is
consistent fir listings. AWS S3 is when s3guard is turned on, but third-party S3 implementations
may also be consistent.
> The committer should be able to ask an FS if it is consistent, something which can be
> * off by default against AWS
> * configured to be one for all buckets (external s3 server)
> * dynamically set when s3guard is enabled.
> This could be done with some predicate isListingConsistent(), or just probed as an FS
option (fs.s3a.consistent.listing) which could be set explicitly in configs, and set dynamically
when s3guard is on. That's actually the simpleset route all round

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

View raw message