hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil Govindan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15663) ABFS: Simplify configuration
Date Tue, 06 Nov 2018 08:49:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16676360#comment-16676360

Sunil Govindan commented on HADOOP-15663:

Updated fixed version to 3.2.0 as HADOOP-15407 branch is merged to trunk

> ABFS: Simplify configuration
> ----------------------------
>                 Key: HADOOP-15663
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15663
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>            Reporter: Thomas Marquardt
>            Assignee: Da Zhou
>            Priority: Major
>             Fix For: 3.2.0
>         Attachments: HADOOP-15663-HADOOP-15407-001.patch, HADOOP-15663-HADOOP-15407-002.patch,
HADOOP-15663-HADOOP-15407-003.patch, HADOOP-15663-HADOOP-15407-004.patch, HADOOP-15663-HADOOP-15407-005.patch
> Configuration for WASB and ABFS is too complex.  The current approach is to use four
files for test configuration. 
> Both WASB and ABFS have basic test configuration which is committed to the repo (azure-test.xml
and azure-bfs-test.xml).  Currently these contain the fs.AbstractFileSystem.[scheme].impl
configuration, but otherwise are empty except for an include reference to a file containing
the endpoint credentials. 
> Both WASB and ABFS have endpoint credential configuration files (azure-auth-keys.xml
and azure-bfs-auth-keys.xml).  These have been added to .gitignore to prevent them from
accidentally being submitted in a patch, which would leak the developers storage account credentials. 
These files contain account names, storage account keys, and service endpoints.
> There is some overlap of the configuration for WASB and ABFS, where they use the same
property name but use different values.  
> 1) Let's reduce the number of test configuration files to one, if possible.
> 2) Let's simplify the account name, key, and endpoint configuration for WASB and ABFS
if possible, but still support the legacy way of doing it, which is very error prone.
> 3) Let's improve error handling, so that typos or misconfiguration are not so difficult
to troubleshoot.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message