hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-13960) Initialize DynamoDBMetadataStore without associated S3AFileSystem
Date Mon, 09 Jan 2017 00:23:58 GMT
Mingliang Liu created HADOOP-13960:

             Summary: Initialize DynamoDBMetadataStore without associated S3AFileSystem
                 Key: HADOOP-13960
                 URL: https://issues.apache.org/jira/browse/HADOOP-13960
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
            Reporter: Mingliang Liu
            Assignee: Mingliang Liu

Per the discussion in [HADOOP-13650], it's helpful to initialize a DynamoDBMetadataStore object
without S3AFileSystem. In the current code, we can achieve this via {{DynamoDBMetadataStore#initialize(Configuration)}}.
However, users still have to provide the associated S3AFileSystem URI in the configuration,
by means of either setting the {{fs.defaultFS}} in configuration file or {{-fs s3://bucket}}
command line parameter. Setting the default FileSystem in configuration seems not necessary
as the command line is to manipulate metadata store, e.g. command line tools on an existing
HDFS cluster.

This JIRA is to track the effort of initializing a DynamoDBMetadataStore without associating
any S3 buckets, so that S3AFileSystem is not needed. Users have to specify in configuration
the DynamoDB endpoints (for region) and table name along with credentials, AWS client settings.

See [~eddyxu] and [~liuml07]'s comments in [HADOOP-13650] for more details.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message