hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.
Date Thu, 15 Dec 2016 00:11:58 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Mingliang Liu updated HADOOP-13455:
    Attachment: HADOOP-13455-HADOOP-13345.003.patch

Thanks [~fabbri] and [~stevel@apache.org]. I like what you guys are doing here. I think early
doc will help us and incoming contributors.

The default value of {{fs.s3a.s3guard.ddb.table.create}} is false in {{core-default.xml}}.
This makes sense in production by asking users to explicitly enable table auto-creation to
avoid unexpected cost. This, however, has slight operation burden to users as she has to either
enable this config along with metadata store impl class, or create DynamoDB table in advance.
The latter is not a problem when importing existing data via command line tool [HADOOP-13650].
Overall, I'm fine with this change.

And we have to remove this config annotation {{@InterfaceAudience.Private}} as it's becoming
public. And to update the javadoc. Moreover I'd suggest we move this from {{S3Guard}} to {{Constants}},
see v3 patch.

In {{DynamoDBMetadataStore}} it was assumed true originally. The failing unit test can be
fixed by setting this config as true in {{TestDynamoDBMetadataStore}}, see the v3 patch.

Interestingly, I also found a bug about the config key. Though we should not create table
unless the config {{fs.s3a.s3guard.ddb.table.create}} is set true, we still have to get the
existing table in {{initialize()}}, before any operation. I will shortly create a separate
JIRA for tracking this and link here.

> S3Guard: Write end user documentation.
> --------------------------------------
>                 Key: HADOOP-13455
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13455
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: HADOOP-13345
>            Reporter: Chris Nauroth
>            Assignee: Aaron Fabbri
>         Attachments: HADOOP-13455-HADOOP-13345-002.patch, HADOOP-13455-HADOOP-13345.001.patch,
> Write end user documentation that describes S3Guard architecture, configuration and usage.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message