hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13311) S3A shell entry point to support commands specific to S3A.
Date Wed, 22 Jun 2016 15:55:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344584#comment-15344584

Chris Nauroth commented on HADOOP-13311:

We can write a more detailed design depending on whether or not we find enough appetite for
this feature.  Depending on the scope defined, this might break up into multiple JIRAs and
possibly use a feature branch.  Here are some initial thoughts.

* s3a diag
** Perform high-level diagnostics.  Print all configuration property values in the fs.s3a.*
namespace.  Make sure not to print secrets though.  Print which credential providers are in
force.  For basic credentials, print if the credentials came from configuration or the credential
provider API.  Print AWS region hosting the bucket.  Print ACLs applied to the bucket.
* s3a fsck
** Scan the bucket looking for unexpected conditions, such as the presence of fake directory
objects that are not really required because of the presence of child files.  Print or optionally
repair.  (Watch out for possibility of data loaded into the bucket externally without using
S3A though.)
* s3a mkfs
** Create the S3 bucket backing the file system.  Add -f option to reformat (delete and recreate)
an existing bucket.  Requires interactive prompt with big scary warnings about deleting data.
* s3a version
** Display the current version of Hadoop, AWS SDK, and relevant transitive dependencies. 
This is potentially useful for users trying to work through version conflicts with their own
code written against the AWS SDK.

> S3A shell entry point to support commands specific to S3A.
> ----------------------------------------------------------
>                 Key: HADOOP-13311
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13311
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Chris Nauroth
>            Priority: Minor
> Create a new {{s3a}} shell entry point.  This can support diagnostic and administrative
commands that are specific to S3A and wouldn't make sense to group under existing scripts
like {{hadoop}} or {{hdfs}}.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message