hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-10433) Key Management Server based on KeyProvider API
Date Tue, 29 Apr 2014 18:06:16 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-10433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Alejandro Abdelnur updated HADOOP-10433:

    Attachment: HADOOP-10433.patch

Andrew, Owen, Larry, thanks for the reviews. New patch addressing them.


* KMS_SILENT does not suppress Tomcat startup messages now.
* KMSMetadata & KMSKeyVersion classes are needed because their parent classes constructors
are protected and per discussion in HADOOP-10431 there was not agreement on making them public.
* Created a KMSClientRESTJSONUtils for client side REST constants/util methods
* Created a KMSServerJSONUtils for server side JSON methods.
* Renamed KMSClientProvider#getKeyMetadata to getMetadata, it is used on the client side on
the corresponding client method.
* on missing audits, we are not missing audits, all exceptions are caught, process and audit
by the KMSAuthenticationFilter. It is a bit weird, but if we do it in the KMServer we still
have to do it in the KMSAuthenticationFilter for uncaught exceptions and for jersey ones.
* made all helper methods private static
* On the validate not throwing exceptions other than IOException, fixed that, thrown the exception
we got in the error if avail in the client, else an IOException
* annotated KMSClientProvider#buildVersionName  as @VisibleForTesting for now
* Fixed the GET/GET_OP thing.
* Added serialVersionUID = 1L to KeyNotFoundException (btw, this is Eclipse non-sense, this
is not used at all outside of java serialization)
* Using a local var cause in KMSCacheKeyProvider not to call ex.getCause() multiple times
* Renamed KMSServer to KMS


* I’ve added a check on the KMSClientProvider#getKeysMetadata() method to throw an IOException
if the resulting URL is longer than 2K. I’d prefer to key the keys in the URL as using a
HEADER would not be really REST.
* I’m not sure where you are suggesting to use _ in fields, in the JSON fields?


* Yes, the <key-name> is part o the JSON payload when creating the key. The create action
is on the collection resource 'keys'. The Response "Location" header has the full URL to the
created resource. AFAIK, this follows common REST practices.

> Key Management Server based on KeyProvider API
> ----------------------------------------------
>                 Key: HADOOP-10433
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10433
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: security
>    Affects Versions: 3.0.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>         Attachments: HADOOP-10433.patch, HADOOP-10433.patch, HADOOP-10433.patch, HADOOP-10433.patch,
HADOOP-10433.patch, HADOOP-10433.patch, HADOOP-10433.patch, HADOOP-10433.patch, HADOOP-10433.patch,
HadoopKMSDocsv2.pdf, KMS-doc.pdf
> (from HDFS-6134 proposal)
> Hadoop KMS is the gateway, for Hadoop and Hadoop clients, to the underlying KMS. It provides
an interface that works with existing Hadoop security components (authenticatication, confidentiality).
> Hadoop KMS will be implemented leveraging the work being done in HADOOP-10141 and HADOOP-10177.
> Hadoop KMS will provide an additional implementation of the Hadoop KeyProvider class.
This implementation will be a client-server implementation.
> The client-server protocol will be secure:
> * Kerberos HTTP SPNEGO (authentication)
> * HTTPS for transport (confidentiality and integrity)
> * Hadoop ACLs (authorization)
> The Hadoop KMS implementation will not provide additional ACL to access encrypted files.
For sophisticated access control requirements, HDFS ACLs (HDFS-4685) should be used.
> Basic key administration will be supported by the Hadoop KMS via the, already available,
Hadoop KeyShell command line tool
> There are minor changes that must be done in Hadoop KeyProvider functionality:
> The KeyProvider contract, and the existing implementations, must be thread-safe
> KeyProvider API should have an API to generate the key material internally
> JavaKeyStoreProvider should use, if present, a password provided via configuration
> KeyProvider Option and Metadata should include a label (for easier cross-referencing)
> To avoid overloading the underlying KeyProvider implementation, the Hadoop KMS will cache
keys using a TTL policy.
> Scalability and High Availability of the Hadoop KMS can achieved by running multiple
instances behind a VIP/Load-Balancer. For High Availability, the underlying KeyProvider implementation
used by the Hadoop KMS must be High Available.

This message was sent by Atlassian JIRA

View raw message