hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-11210) Enhance key rolling to be atomic
Date Tue, 31 Jan 2017 23:20:52 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Xiao Chen updated HDFS-11210:
    Attachment: HDFS-11210.02.patch

Thanks for the review Andrew, really appreciate the quality review of yours. :)

Good call to {{invalidate}} cache, don't know what I was thinking when naming it...

bq. High-level, is KeyProvider#clearCache intended for use by end-users?
Since {{invalidateCache}} is added to KMS, and the keyprovider API, end-users can definitely
use it. I try to hide this detail as much as possible, but agree {{hadoop key}} command addition
would be handy, and no harm. So added this and a test in patch 2.

All comments addressed I think. Also added a new test in {{TestKMS}} with mock, to fail-before-pass-after
the invalidation. The rollover draining test is race by nature, so IMHO the separate mock
testing would be clearer.

> Enhance key rolling to be atomic
> --------------------------------
>                 Key: HDFS-11210
>                 URL: https://issues.apache.org/jira/browse/HDFS-11210
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: encryption, kms
>    Affects Versions: 2.6.5
>            Reporter: Xiao Chen
>            Assignee: Xiao Chen
>         Attachments: HDFS-11210.01.patch, HDFS-11210.02.patch
> To support re-encrypting EDEK, we need to make sure after a key is rolled, no old version
EDEKs are used anymore. This includes various caches when generating EDEK.
> This is not true currently, simply because no such requirements / necessities before.
> This includes
> - Client Provider(s), and corresponding cache(s).
> When LoadBalancingKMSCP is used, we need to clear all KMSCPs.
> - KMS server instance(s), and corresponding cache(s)
> When KMS HA is configured with multiple KMS instances, only 1 will receive the {{rollNewVersion}}
request, we need to make sure other instances are rolled too.
> - The Client instance inside NN(s), and corresponding cache(s)
> When {{hadoop key roll}} is succeeded, the client provider inside NN should be drained

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message