hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12172) Reduce EZ lookup overhead
Date Mon, 24 Jun 2019 19:23:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16871718#comment-16871718

Wei-Chiu Chuang commented on HDFS-12172:

The patch was never merged, but it looks like a good one. I rebased the code and let's see
what it says.

The biggest difference is that I kept {{EncryptionZoneManager #getFullPathName()}} because
the reencrypt code depends on it, and I don't understand this part of the code well enough
to optimize it accordingly. (yes, I reviewed the reencrypt implementation but don't remember
the details like this) It seems like it'll require a sizable change in the reencrypt code
to make that work.

> Reduce EZ lookup overhead
> -------------------------
>                 Key: HDFS-12172
>                 URL: https://issues.apache.org/jira/browse/HDFS-12172
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.7.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Major
>         Attachments: HDFS-12172.002.patch, HDFS-12172.003.patch, HDFS-12172.01.patch,
> A number of inefficiencies exist in EZ lookups.  These are amplified by frequent operations
like list status.  Once one encryption zone exists, all operations take the performance penalty.
> Ex. Operations should not perform redundant lookups.  EZ path reconstruction should be
lazy since it's not required in the common case.  Renames do not need to reallocate new IIPs
to check parent dirs for EZ.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message