ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keta Patel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-18071) Ambari Files View needs to have ability to load security configurations
Date Wed, 31 Aug 2016 23:22:20 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-18071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453691#comment-15453691
] 

Keta Patel commented on AMBARI-18071:
-------------------------------------

CAUSE:
The error of "No KeyProvider is configured" is seen only for those cases when the HDFS uses
DistributedFileSystem for its communication. When HDFS uses WebHDFSFileSystem for communication,
this error is not seen and the Ambari View instance is able to open the files in the encrypted
zones. 

Why Ambari Views use either Distributed or WebHDFS file systems is explained below:
Ambari views can be created using one of the 3 modes of configuration:
1. Local cluster
2. Remote cluster
3. Custom configuration (no cluster is associated here).

The HDFS works through abstraction. For Ambari Views, the actual file system used during execution
depends on whether the view instance was created using a Local/Remote cluster or using Custom
configuration. For instances created using Local/Remote cluster, HDFS uses Distributed File
System and for instances created using Custom configuration, HDFS uses WebHDFSFileSystem.
WebHDFSFileSystem is an integrated part of the HDFS ecosystem. It is aware of all the HDFS
configuration. For this reason, when a KMS is configured in HDFS, WebHDFSFileSystem is aware
of the KeyProvider and no special config mapping is needed. Thus, even the view instance created
using Custom configuration doesn't need any special configuration and can talk to the Encryption
Zones successfully. 

However, for view instances created using Local/Remote cluster configuration, HDFS uses the
Distributed FileSystem. This Distributed FileSystem works as an HDFS client and hence, is
not fully aware of all the HDFS configuration. We need to explicitly provide HDFS properties
like "dfs.encryption.key.provider.uri" to these ambari view instances to provide details of
the KeyProvider. The proposed fix helps in providing this property value to the view as follows.


FIX:
====
The proposed fix (attached as "AMBARI-18071.patch") checks if the current view instance configuration
has any cluster associated in its context. If there is an associated cluster then the instance
has a Local/Remote cluster configuration and needs to be provided with the HDFS KeyProvider
information. Otherwise, the WebHDFSFileSystem will take care of the KeyProvider if KMS is
configured.

To provide the property information, the parseProperties() in ConfigurationBuilder.java looked
best as we also set the defaultFS property here. If a cluster is associated with the context,
and if the property "dfs.encryption.key.provider.uri" is not null, then this property is set
in the Configuration object and thus made available to Distributed file system of HDFS.
The Ambari VIew instance works successfully with both Local and Remote configurations.

**One more point to note in the configuration aspect is the addition of proxyuser to the kms-site.xml
for the ambari-server daemon. Without this proxyuser even the custom configuration will not
work. (I had installed hadoop's KMS on the ambari-server manually)

> Ambari Files View needs to have ability to load security configurations
> -----------------------------------------------------------------------
>
>                 Key: AMBARI-18071
>                 URL: https://issues.apache.org/jira/browse/AMBARI-18071
>             Project: Ambari
>          Issue Type: Improvement
>          Components: contrib
>    Affects Versions: trunk
>            Reporter: Keta Patel
>            Assignee: Keta Patel
>
> When HDFS is configured with Encryption Zones, Files View to browser files will give
"No KeyProvider" error.
> Steps to reproduce this issue:
> 1. Configure an encrypted zone in HDFS (Transparent Data Encryption) following the link
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_hdfs_admin_tools/content/ch_configuring_hdfs_encryption.html
> 2. Create a Files View instance and provide a user/group the privilege to use the instance.
> 3. Log into the Ambari console as the user with the Files View permission.
> 4. Open the Files View instance.
> 5. Go to the folder which is configured as an encrypted zone.
> 6. Try to open an existing file in this folder.
> 7. This throws an error - java.io.IOException: No KeyProvider is configured, cannot access
an encrypted file. 
> 8. When trying through the shell, opening this file works.
> This happens because Files View doesn't have enough configuration set to browse secured
zone. Files view doesn't even provide an option to add these configurations.This is why we
see errors "No KeyProvider is configured, cannot access an encrypted file", to work around
this, you could download client configuration from HDFS service tab, and copy the core-site.xml
and hdfs-site.xml files to /etc/ambari-server/conf, then restart ambari-server. After this,
the user is able to open the file in the encrypted zone.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message