Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 21 Dec 2016 23:12:58 +0000 (UTC)
From: "Mingliang Liu (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13025361.1480802522000.576302.1482361978650@Atlassian.JIRA>
In-Reply-To: <JIRA.13025361.1480802522000@Atlassian.JIRA>
References: <JIRA.13025361.1480802522000@Atlassian.JIRA> <JIRA.13025361.1480802522807@arcas>
Subject: [jira] [Comment Edited] (HADOOP-13863) Azure: Add a new SAS key
 mode for WASB.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 21 Dec 2016 23:13:00 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-13863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15768441#comment-15768441 ] 

Mingliang Liu edited comment on HADOOP-13863 at 12/21/16 11:12 PM:
-------------------------------------------------------------------

The patch looks good to me overall.

# In {{core-default.xml}},
{quote}
If fs.azure.secure flag is set to false, this flag has no effect.
{quote}
I think here we should refer to {{fs.azure.secure.mode}}?
# Should we also add the default value of {{fs.azure.sas.expiry.period}} to the {{core-default.xml}} file? Also please document that the config key supports multiple time unit suffix(case insensitive), as described in dfs.heartbeat.interval.
# {{The variable is declared public for visibility in Tests}} this comment can be replaced with annotation {{@VisibleForTesting}}
# Can you file separate JIRA to address the javac errors? I think they're mostly not related to this patch. Can you confirm that?
# The follow code throws an exception. Is it actually an assert (bug in implementation code instead of usage/configuration)? I'm not very sure about this.
{code:title=AzureNativeFileSystemStore#connectToAzureStorageInSecureMode()}
839	    if (!(this.storageInteractionLayer instanceof SecureStorageInterfaceImpl)) {
840	      throw new AzureException("Invalid State reached."
841	          + " connectToAzureStorageInSASKeyMode called when"
842	          + " SASStorageInterfaceImpl not used");
843	    }
{code}
# In class {{}}, can you add comment for the following code? It's not very straightforward; other code in this method is documented well.
{code:title=AzureNativeFileSystemStore#createAzureStorageSession()}
984	      if (useSecureMode) {
985	        connectToAzureStorageInSecureMode(accountName, containerName, sessionUri);
986	        return;
987	      }
{code}
# {{WasbRemoteCallHelper}} be package local?


was (Author: liuml07):
The patch looks good to me overall.

# In {{core-default.xml}},
{quote}
If fs.azure.secure flag is set to false, this flag has no effect.
{quote}
I think here we should refer to {{fs.azure.secure.mode}}?
# Should we also add the default value of {{fs.azure.sas.expiry.period}} to the {{core-default.xml}} file? Also please document that the config key supports multiple time unit suffix(case insensitive), as described in dfs.heartbeat.interval.
# {{   * The variable is declared public for visibility in Tests}} this comment can be replaced with annotation {{@VisibleForTesting}}
# Can you file separate JIRA to address the javac errors? I think they're mostly not related to this patch. Can you confirm that?
# The follow code throws an exception. Is it actually an assert (bug in implementation code instead of usage/configuration)? I'm not very sure about this.
{code:title=AzureNativeFileSystemStore#connectToAzureStorageInSecureMode()}
839	    if (!(this.storageInteractionLayer instanceof SecureStorageInterfaceImpl)) {
840	      throw new AzureException("Invalid State reached."
841	          + " connectToAzureStorageInSASKeyMode called when"
842	          + " SASStorageInterfaceImpl not used");
843	    }
{code}
# In class {{}}, can you add comment for the following code? It's not very straightforward; other code in this method is documented well.
{code:title=AzureNativeFileSystemStore#createAzureStorageSession()}
984	      if (useSecureMode) {
985	        connectToAzureStorageInSecureMode(accountName, containerName, sessionUri);
986	        return;
987	      }
{code}
# {{WasbRemoteCallHelper}} be package local?

> Azure: Add a new SAS key mode for WASB.
> ---------------------------------------
>
>                 Key: HADOOP-13863
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13863
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: azure, fs/azure
>    Affects Versions: 2.8.0
>            Reporter: Dushyanth
>            Assignee: Dushyanth
>         Attachments: HADOOP-13863.001.patch, HADOOP-13863.002.patch, HADOOP-13863.003.patch, HADOOP-13863.004.patch, HADOOP-13863.005.patch, HADOOP-13863.006.patch, Proposal-Document.pdf
>
>
> Current implementation of WASB, only supports Azure storage keys and SAS key being provided via org.apache.hadoop.conf.Configuration, which results in these secrets residing in the same address space as the WASB process and providing complete access to the Azure storage account and its containers. Added to the fact that WASB does not inherently support ACL's, WASB is its current implementation cannot be securely used for environments like secure hadoop cluster. This JIRA is created to add a new mode in WASB, which operates on Azure Storage SAS keys, which can provide fine grained timed access to containers and blobs, providing a segway into supporting WASB for secure hadoop cluster.
> More details about the issue and the proposal are provided in the design proposal document.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org