hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-11885) createEncryptionZone should not block on initializing EDEK cache
Date Wed, 31 May 2017 01:59:04 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Wang updated HDFS-11885:
-------------------------------
    Attachment: HDFS-11885.001.patch

Here's a patch which breaks out the EDEKCacheLoader into its own class, and uses it to run
the EDEK warm up in the background on createZone.

Rushabh, thanks for the offer. createZone still queries the KMS synchronously in {{ensureKeyIsInitialized}}
to see if the key exists. Your proposed patch would still help with that. This JIRA is just
to help with slow generation of EDEKs.

> createEncryptionZone should not block on initializing EDEK cache
> ----------------------------------------------------------------
>
>                 Key: HDFS-11885
>                 URL: https://issues.apache.org/jira/browse/HDFS-11885
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: encryption
>    Affects Versions: 2.6.5
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>            Priority: Critical
>         Attachments: HDFS-11885.001.patch
>
>
> When creating an encryption zone, we call {{ensureKeyIsInitialized}}, which calls {{provider.warmUpEncryptedKeys(keyName)}}.
This is a blocking call, which attempts to fill the key cache up to the low watermark.
> If the KMS is down or slow, this can take a very long time, and cause the createZone
RPC to fail with a timeout.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message