hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Marquardt (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-15663) ABFS: Simplify configuration
Date Thu, 09 Aug 2018 18:37:00 GMT
Thomas Marquardt created HADOOP-15663:

             Summary: ABFS: Simplify configuration
                 Key: HADOOP-15663
                 URL: https://issues.apache.org/jira/browse/HADOOP-15663
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/azure
            Reporter: Thomas Marquardt

Configuration for WASB and ABFS is too complex.  The current approach is to use four files
for test configuration. 

Both WASB and ABFS have basic test configuration which is committed to the repo (azure-test.xml
and azure-bfs-test.xml).  Currently these contain the fs.AbstractFileSystem.[scheme].impl
configuration, but otherwise are empty except for an include reference to a file containing
the endpoint credentials. 

Both WASB and ABFS have endpoint credential configuration files (azure-auth-keys.xml and azure-bfs-auth-keys.xml). 
These have been added to .gitignore to prevent them from accidentally being submitted in a
patch, which would leak the developers storage account credentials.  These files contain
account names, storage account keys, and service endpoints.

There is some overlap of the configuration for WASB and ABFS, where they use the same property
name but use different values.  

1) Let's reduce the number of test configuration files to one, if possible.

2) Let's simplify the account name, key, and endpoint configuration for WASB and ABFS if possible,
but still support the legacy way of doing it, which is very error prone.

3) Let's improve error handling, so that typos or misconfiguration are not so difficult to

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

View raw message