hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Charles Lamb (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones
Date Wed, 27 Aug 2014 16:33:58 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Charles Lamb updated HDFS-6951:

    Attachment: HDFS-6951.001.patch


Nice find. The problem is that when we create a compact representation of xattrs to store
in the inode we only have 2 bits for the xattr namespace. That worked fine when there were
only 4 xattr namespaces (user, trusted, security, system), but when the 'raw' namespace was
added, the change did not take this into account. The problem you discovered occurs because
the EZ is "indicated" by a raw.hdfs.crypto... xattr. When that xattr was recovered during
namespace loading it came back as user.hdfs.crypto... instead of raw.hdfs.crypto... A mess

Fortunately in the compact xattr representation, there are a handful of unused (reserved for
future use) bits. The attached patch leaves the existing two NS bits and 24 name bits in place
and then uses one of the reserved bits as a third NS bit. This should be backward compatible
since that reserved bit will (presumably) be 0 in existing inodes.

I've also added your test (thank you!) to TestEncryptionZones.java as well as a new xattr-specific
test for this particular case.

[~andrew.wang], please take a look. BTW, I removed the assert from EZM because during edit
log recovery that method actually gets called without the fsd lock being held.

> Saving namespace and restarting NameNode will remove existing encryption zones
> ------------------------------------------------------------------------------
>                 Key: HDFS-6951
>                 URL: https://issues.apache.org/jira/browse/HDFS-6951
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: encryption
>    Affects Versions: 3.0.0
>            Reporter: Stephen Chu
>            Assignee: Charles Lamb
>             Fix For: 3.0.0
>         Attachments: HDFS-6951-testrepo.patch, HDFS-6951.001.patch
> Currently, when users save namespace and restart the NameNode, pre-existing encryption
zones will be wiped out.
> I could reproduce this on a pseudo-distributed cluster:
> * Create an encryption zone
> * List encryption zones and verify the newly created zone is present
> * Save the namespace
> * Kill and restart the NameNode
> * List the encryption zones and you'll find the encryption zone is missing
> I've attached a test case for {{TestEncryptionZones}} that reproduces this as well. Removing
the saveNamespace call will get the test to pass.

This message was sent by Atlassian JIRA

View raw message