hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12357) Let NameNode to bypass external attribute provider for special user
Date Wed, 06 Sep 2017 00:10:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16154605#comment-16154605
] 

Manoj Govindassamy commented on HDFS-12357:
-------------------------------------------

Thanks for working on this [~yzhangal]. Thanks [~chris.douglas] for the valuable comments
and the alternative proposals. Much appreciated.

My review comments for HDFS-12357.001b.patch.

1. {{FSDirectory.java}}
-- {{getUserFilteredAttributeProvider}} -- line 401 to 407 can be simplified in a single if
block
{noformat}
if (ugi == null || isUserBypassingExtAttrProvider(ugi.getUserName()) {
   return null;
}
{noformat}

-- {{initUsersToBypassExtProvider()}} : an user list like "a, b, " can trip the code to add
a null object to the {{usersToBypassExtAttrProvider}}. Probably we want to verify the trimmed
user before adding it to the bypass list.

2. {{hdfs-default.xml}}
-- "..for whom the external attributes provider will be bypassed" - This config description
can have more details like what bypass would mean for the user operations. Is that only for
permission checking or other operations etc.,

3. {{TestINodeAttributeProvider}}
-- Can you please add one non-bypassed user "u4" to the test list of users in line 239? Basically
a true negative case.
-- Check style issues

4. Probably the patch can be renamed to HDFS-12357.006.patch so that any new reviewers looking
at this jira can go straight to the latest patch instead of suffix versions in the older patch.

> Let NameNode to bypass external attribute provider for special user
> -------------------------------------------------------------------
>
>                 Key: HDFS-12357
>                 URL: https://issues.apache.org/jira/browse/HDFS-12357
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>         Attachments: HDFS-12357.001a.patch, HDFS-12357.001b.patch, HDFS-12357.001.patch,
HDFS-12357.002.patch, HDFS-12357.003.patch, HDFS-12357.004.patch, HDFS-12357.005.patch
>
>
> This is a third proposal to solve the problem described in HDFS-12202.
> The problem is, when we do distcp from one cluster to another (or within the same cluster),
in addition to copying file data, we copy the metadata from source to target. If external
attribute provider is enabled, the metadata may be read from the provider, thus provider data
read from source may be saved to target HDFS. 
> We want to avoid saving metadata from external provider to HDFS, so we want to bypass
external provider when doing the distcp (or hadoop fs -cp) operation.
> Two alternative approaches were proposed earlier, one in HDFS-12202, the other in HDFS-12294.
The proposal here is the third one.
> The idea is, we introduce a new config, that specifies a special user (or a list of users),
and let NN bypass external provider when the current user is a special user.
> If we run applications as the special user that need data from external attribute provider,
then it won't work. So the constraint on this approach is, the special users here should not
run applications that need data from external provider.
> Thanks [~asuresh] for proposing this idea and [~chris.douglas], [~daryn], [~manojg] for
the discussions in the other jiras. 
> I'm creating this one to discuss further.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message