hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Charles Lamb (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10919) Copy command should preserve raw.* namespace extended attributes
Date Tue, 12 Aug 2014 01:26:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093611#comment-14093611

Charles Lamb commented on HADOOP-10919:


There are three scenarios. 

(1) An administrator who does not have access to the keys in the KMS would use the /.reserved/raw
prefix on src and dest:

distcp /.reserved/raw/src /.reserved/raw/dest

The /.reserved/raw is the only interface that exposes the raw.* xattrs holding the encryption
metadata. This allows the raw.* xattrs to be preserved on the dest as well as to copy the
files without decrypting them. This scenario assumes that an ez has been set up on dest. As
you suggested, it would be a good idea to check that the dest is actually an ez.

(2) A non-admin user who has access to some subset of files in an ez could use the non-/.reserved/raw
prefix and copy a hierarchy from one ez to another. In that case, the raw.* xattrs from the
src ez would not be preserved. This scenario assumes that the dest ez is already set up. Of
course the dest files will have new keys associated with them since they'll be new copies.

(3) Neither src or dst has /.reserved/raw and one or the other of src/dest is not an ez. It
is not necessary to have the target also be an ez. The use case would be that the user wants
to copy a subset of the ez into/out-of a non-encrypted file system. distcp without the /.reserved/raw
prefix could be used for this.

Does this all make sense?

> Copy command should preserve raw.* namespace extended attributes
> ----------------------------------------------------------------
>                 Key: HADOOP-10919
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10919
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 3.0.0
>            Reporter: Charles Lamb
>            Assignee: Charles Lamb
>             Fix For: fs-encryption (HADOOP-10150 and HDFS-6134)
>         Attachments: HADOOP-10919.001.patch, HADOOP-10919.002.patch
> Refer to the doc attached to HDFS-6509 for background.
> Like distcp -p (see MAPREDUCE-6007), the copy command also needs to preserve extended
attributes in the raw.* namespace by default whenever the src and target are in /.reserved/raw.
To not preserve raw xattrs, don't specify /.reserved/raw in either the src or target. 

This message was sent by Atlassian JIRA

View raw message