hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Deleted] (HADOOP-13876) S3Guard: better support for multi-bucket access including read-only
Date Thu, 19 Jan 2017 17:55:27 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Steve Loughran updated HADOOP-13876:
------------------------------------
    Comment: was deleted

(was: HADOOP-13985 versioning patch 002
* adds a created timestamp to the version field as well
* fixes AbstractFileSystem to unwrap IOEs raised in FS construction better, so preserving
error text
* reinstates table.describe() on initTable, rather than just the getItem. Why? getItem() seemed
to trogger table creation, which would then fail as there was no version field. This is is
something odd that we really need to understand. (Repeatable BTW: delete the table in the
AWS console, run a test, see the stack trace)

tested s3 ireland with {{-Ds3guard -Dparallel-tests -DtestsThreadCount=6 -Ddynamo -Dauth -Dtest=moo
 -Dscale -Dfs.s3a.scale.test.huge.filesize=6M}})

> S3Guard: better support for multi-bucket access including read-only
> -------------------------------------------------------------------
>
>                 Key: HADOOP-13876
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13876
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: HADOOP-13345
>            Reporter: Aaron Fabbri
>            Assignee: Mingliang Liu
>         Attachments: HADOOP-13876-HADOOP-13345.000.patch
>
>
> HADOOP-13449 adds support for DynamoDBMetadataStore.
> The code currently supports two options for choosing DynamoDB table names:
> 1. Use name of each s3 bucket and auto-create a DynamoDB table for each.
> 2. Configure a table name in the {{fs.s3a.s3guard.ddb.table}} parameter.
> One of the issues is with accessing read-only buckets.  If a user accesses a read-only
bucket with credentials that do not have DynamoDB write permissions, they will get errors
when trying to access the read-only bucket.  This manifests causes test failures for {{ITestS3AAWSCredentialsProvider}}.
> Goals for this JIRA:
> - Fix {{ITestS3AAWSCredentialsProvider}} in a way that makes sense for the real use-case.
> - Allow for a "one DynamoDB table per cluster" configuration with a way to chose which
credentials are used for DynamoDB.
> - Document limitations etc. in the s3guard.md site doc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message