hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaoyu Yao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7256) Encryption Key created in Java Key Store after Namenode start unavailable for EZ Creation
Date Fri, 17 Oct 2014 05:40:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14174744#comment-14174744

Xiaoyu Yao commented on HDFS-7256:

Thanks [~hitliuyi] for the detail explanation.  I configured my test environment based on
HDFS-6134 proposal: https://issues.apache.org/jira/secure/attachment/12660368/HDFSDataatRestEncryption.pdf.

Can you point me the link to fs-encryption/KMS user doc if there is a different one?

I do have a KMS setup with JavaKeyStoreProvider pointing to the same java key store file.

Based on your suggestion, I just switch to use 'kms://http@localhost:16000/kms' instead of
the java key store file 
'jceks://file/home/hadoop/kms.keystore' directly for the 'dfs.encryption.key.provider.uri'
in hdfs-site.xml and 'hadoop.security.crypto.jce.provider' in core-site.xml.

Below I have two follow up questions when executing the the 'hadoop key' command after the
change. Can you confirm if these are expected or not?

1. Have to specify -provider explicitly even though hadoop.security.crypto.jce.provider='kms://http@localhost:16000/kms'
is configured in core-site.xml.

hadoop@hadoopdev:~/deploy$ hadoop/bin/hadoop key list
There are no non-transient KeyProviders configured.
Use the -provider option to specify a provider. If you
want to list a transient provider then you must use the
-provider argument.

2. Keys are returned with -provider specified but WARN message is logged in kms.log on Anonymous
request. My understanding is that KMS should proxy user 'hadoop' based the proxy user setting
below. Do I miss anything?
hadoop@hadoopdev:~/deploy$ hadoop/bin/hadoop key list -provider kms://http@localhost:16000/kms
Listing keys for KeyProvider: KMSClientProvider[http://localhost:16000/kms/v1/]

2014-10-16 22:08:38,386 WARN  AuthenticationFilter - Authentication exception: Anonymous requests
are disallowed
org.apache.hadoop.security.authentication.client.AuthenticationException: Anonymous requests
are disallowed
        at org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:184)
        at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:330)
        at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507)
        at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
        at java.lang.Thread.run(Thread.java:745)

The client runs with user 'hadoop'. The proxyuser and delegation token(use default) are set
up in kms-site.xml. 
  <!-- proxyuser configuration for user named:  hadoop-->

> Encryption Key created in Java Key Store after Namenode start unavailable for EZ Creation

> ------------------------------------------------------------------------------------------
>                 Key: HDFS-7256
>                 URL: https://issues.apache.org/jira/browse/HDFS-7256
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: encryption, security
>    Affects Versions: 2.6.0
>            Reporter: Xiaoyu Yao
> Hit an error on "RemoteException: Key ezkey1 doesn't exist." when creating EZ with a
Key created after NN starts.
> Briefly check the code and found that the KeyProivder is loaded by FSN only at the NN
start. My work around is to restart the NN which triggers the reload of Key Provider. Is this
> Repro Steps:
> Create a new Key after NN and KMS starts
> hadoop/bin/hadoop key create ezkey1 -size 256 -provider jceks://file/home/hadoop/kms.keystore
> List Keys
> hadoop@SaturnVm:~/deploy$ hadoop/bin/hadoop key list -provider jceks://file/home/hadoop/kms.keystore
> Listing keys for KeyProvider: jceks://file/home/hadoop/kms.keystore
> ezkey1 : cipher: AES/CTR/NoPadding, length: 256, description: null, created: Thu Oct
16 18:51:30 EDT 2014, version: 1, attributes: null
> key2 : cipher: AES/CTR/NoPadding, length: 128, description: null, created: Tue Oct 14
19:44:09 EDT 2014, version: 1, attributes: null
> key1 : cipher: AES/CTR/NoPadding, length: 128, description: null, created: Tue Oct 14
17:52:36 EDT 2014, version: 1, attributes: null
> Create Encryption Zone
> hadoop/bin/hdfs dfs -mkdir /Ez1
> hadoop@SaturnVm:~/deploy$ hadoop/bin/hdfs crypto -createZone -keyName ezkey1 -path /Ez1
> RemoteException: Key ezkey1 doesn't exist.

This message was sent by Atlassian JIRA

View raw message