hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Charles Lamb (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6509) create a /.reserved/raw filesystem namespace
Date Mon, 28 Jul 2014 00:11:39 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Charles Lamb updated HDFS-6509:
-------------------------------

    Attachment: HDFS-6509.002.patch

[~andrew.wang],

Thanks for your review. I've attached a .002 version which I believe addresses your comments.

FSN and FSDir:

bq. You can just do isReservedRawName(src) in place rather than declaring a new variable if
it's not being reused. Very cheap function too.

But since resolvePathname changes src to remove /.reserved/raw, I needed to call iiRRN prior
to the resolvePathname call. It's a moot point now that I've fixed the logAudit calls to log
the /.reserved/raw pathnames.

bq. Lines longer than 80 chars

I fixed the one in constructRemainingPath (I left that in the first diffs since the diffs
didn't show that particular line as a change). Were there others besides that one? The comments
for resolvePath ran right up to 80, but I don't think they were over 80.

bq. Need <p/> to line break in a javadoc comment.

Fixed.

bq. I shy away from the term "namespace" for raw, can we just say "directory"? It is just
a directory after all.

Fixed.

bq. FSDirectory, not a fan of even mildly complicated ternary statements. Can we turn these
into ifs, or at least not inline them into method calls?

Fixed. There was only one inside a method call. I moved it outside. The other ternary statements
didn't look too complicated so I didn't change them, but if there still some that you don't
like, let me know.

bq. Calling up to FSNamesystem from FSDirectory is a no-no, since we want to separate FSD
and FSN locks and promote a lock hierarchy. Can we do the superuser check in FSNamesystem?

I've turned all of the FSD.resolvePath() calls in FSN into a new FSN#resolvePath. This looked
better than adding a call to  to checkAccessForReservedRaw(src) before every call to FSD#resolvePath.

bq. Stripping the prefix off is scary because we lose context. At the very least, we should
log the original path in the audit log. Various log messages will be more confusing. I think
using the stripped path is okay in the edit log.

Agree. I've made all of the relevant logAuditEvent calls use the pre-resolvePathname path
rather than the non-/.reserved/raw pathname.

bq. Some javadocs needs updating for the new isRawPath parameter. 

I added @param isRawPath to FSD#getFileInfo. I think that was the only one.

Test:

bq. Helper methods look duplicated from TestEncryptionZones. Let's extract the common code
somewhere like DFSTestUtil or GenericTestUtils.

Done.

bq. Why is the special casing necessary in TestDFSUtil for mkdirs? If mkdirs behaves incorrectly
on a raw path, then we should fix mkdirs. A normal user could call mkdirs on a raw path after
all.

Artifact. Removed.

bq. Can we name the test "TestReservedRawPaths"?

Done.

bq. The ".." tests, let's do some tests where the ".." is in between .reserved and raw.

Done.

bq. There's a dfsadmin defined for the class, why create a new one in each test? The dfsAdmin
in testRelativePathnames also isn't used.

Fixed.

bq. Why can't we list /.reserved? Seems like it should work.

That feels a bit ugly. While I agree that it would be nice to be able to list /.reserved and
have it show raw and .inodes, we'd have to create and return a DirectoryListing with a pair
of HdfsFileStatus instances, one for raw, one for .inodes. There would be a bunch of special
case code. If you feel strongly about this, then I'd rather do this work in a followon Jira.

bq. Should also assert the entry that we get back is the entry we expect, not just the length.

Fixed.


> create a /.reserved/raw filesystem namespace
> --------------------------------------------
>
>                 Key: HDFS-6509
>                 URL: https://issues.apache.org/jira/browse/HDFS-6509
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: security
>    Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134)
>            Reporter: Charles Lamb
>            Assignee: Charles Lamb
>         Attachments: HDFS-6509.001.patch, HDFS-6509.002.patch, HDFS-6509distcpandDataatRestEncryption-2.pdf,
HDFS-6509distcpandDataatRestEncryption-3.pdf, HDFS-6509distcpandDataatRestEncryption.pdf
>
>
> This is part of the work for making distcp work with Data at Rest Encryption. Per the
attached document, create a /.reserved/raw HDFS filesystem namespace that allows access to
the encrypted bytes of a file.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message