hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-9405) When starting a file, NameNode should generate EDEK in a separate thread
Date Wed, 09 Mar 2016 17:33:40 GMT

     [ https://issues.apache.org/jira/browse/HDFS-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Xiao Chen updated HDFS-9405:
    Attachment: HDFS-9405.01.patch

Patch 1 attached warms up the cache on NN startup/failover. IMHO daemon might be an overkill,
so a thread is simply launched to do the work.

For the rare case that KMS/backing key provider is down, ideally we may want to warm up as
well. Unfortunately we don't have a good way to actively identify this from hdfs. It may be
helpful to add a {{dfsadmin}} command to do so?

For the change in RPC, I feel it is big and incompatible. Also seems above discussions are
all about the warm up, so I propose not to make this change here. Thoughts?

> When starting a file, NameNode should generate EDEK in a separate thread
> ------------------------------------------------------------------------
>                 Key: HDFS-9405
>                 URL: https://issues.apache.org/jira/browse/HDFS-9405
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: encryption, namenode
>    Affects Versions: 2.7.1
>            Reporter: Zhe Zhang
>            Assignee: Xiao Chen
>         Attachments: HDFS-9405.01.patch
> {{generateEncryptedDataEncryptionKey}} involves a non-trivial I/O operation to the key
provider, which could be slow or cause timeout. It should be done as a separate thread so
as to return a proper error message to the RPC caller.

This message was sent by Atlassian JIRA

View raw message