hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12172) Reduce EZ lookup overhead
Date Thu, 03 Aug 2017 00:10:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111958#comment-16111958

Xiao Chen commented on HDFS-12172:

Thanks for the work here [~daryn]! Looks pretty good overall.

A few nits below:
- Could you add some javadocs to the {{EncryptionZoneManager.java#getEncryptionZoneInt(INodesInPath
iip, int offset)}} method? Without looking into the method body, I wasn't sure how the offset
should be passed in. Good to also add some assertions that {{offset < 0 && offset
> iip.length()}} too.
- {{FSDirEncryptionZoneOp#getFileEncryptionInfo(FSDirectory dir, EncryptionZoneInt zone, EncryptionKeyInfo
ezInfo)}} can get rid of the first {{dir}} param now.
- javadoc on {{FSDirEncryptionZoneOp#getEncryptionKeyInfo}} needs updating

+1 pending the above.

> Reduce EZ lookup overhead
> -------------------------
>                 Key: HDFS-12172
>                 URL: https://issues.apache.org/jira/browse/HDFS-12172
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.7.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: HDFS-12172.patch
> A number of inefficiencies exist in EZ lookups.  These are amplified by frequent operations
like list status.  Once one encryption zone exists, all operations take the performance penalty.
> Ex. Operations should not perform redundant lookups.  EZ path reconstruction should be
lazy since it's not required in the common case.  Renames do not need to reallocate new IIPs
to check parent dirs for EZ.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message