hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aengin...@apache.org
Subject [19/48] hadoop git commit: HADOOP-12947. Update documentation Hadoop Groups Mapping to add static group mapping, negative cache. Contributed by Wei-Chiu Chuang.
Date Mon, 28 Mar 2016 17:35:47 GMT
HADOOP-12947. Update documentation Hadoop Groups Mapping to add static group mapping, negative
cache. Contributed by Wei-Chiu Chuang.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/938222b2
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/938222b2
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/938222b2

Branch: refs/heads/HDFS-7240
Commit: 938222b2e20791d6eb410e36f8fec3e1520e1078
Parents: 7381550
Author: Andrew Wang <wang@apache.org>
Authored: Wed Mar 23 14:57:14 2016 -0700
Committer: Andrew Wang <wang@apache.org>
Committed: Wed Mar 23 14:57:14 2016 -0700

----------------------------------------------------------------------
 .../src/site/markdown/GroupsMapping.md          | 26 +++++++++++---------
 1 file changed, 15 insertions(+), 11 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/938222b2/hadoop-common-project/hadoop-common/src/site/markdown/GroupsMapping.md
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/GroupsMapping.md b/hadoop-common-project/hadoop-common/src/site/markdown/GroupsMapping.md
index 5a67bd1..a742029 100644
--- a/hadoop-common-project/hadoop-common/src/site/markdown/GroupsMapping.md
+++ b/hadoop-common-project/hadoop-common/src/site/markdown/GroupsMapping.md
@@ -15,14 +15,7 @@
 Hadoop Groups Mapping
 ===================
 
-* [Hadoop Groups Mapping](#Hadoop_Groups_Mapping)
-    * [Overview](#Overview)
-    * [LDAP Groups Mapping](#LDAP_Groups_Mapping)
-        * [Active Directory](#Active_Directory)
-        * [POSIX Groups](#POSIX_Groups)
-        * [SSL](#SSL)
-    * [Composite Groups Mapping](#Composite_Groups_Mapping)
-        * [Multiple group mapping providers configuration sample](#Multiple_group_mapping_providers_configuration_sample)
+<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
 
 Overview
 --------
@@ -61,6 +54,17 @@ For HDFS, the mapping of users to groups is performed on the NameNode.
Thus, the
 
 Note that HDFS stores the user and group of a file or directory as strings; there is no conversion
from user and group identity numbers as is conventional in Unix.
 
+Static Mapping
+--------
+It is possible to statically map users to groups by defining the mapping in `hadoop.user.group.static.mapping.overrides`
in the format `user1=group1,group2;user2=;user3=group2`.
+This property overrides any group mapping service provider. If a user's groups are defined
in it, the groups are returned without more lookups; otherwise, the service provider defined
in `hadoop.security.group.mapping` is used to look up the groups. By default, `dr.who=;` is
defined, so the fake user dr.who will not have any groups.
+
+Caching/Negative caching
+--------
+Since the group mapping resolution relies on external mechanisms, the NameNode performance
may be impacted. To reduce the impact due to repeated lookups, Hadoop caches the groups returned
by the service provider. The cache invalidate is configurable via `hadoop.security.groups.cache.secs`,
and the default is 300 seconds.
+
+To avoid spamming NameNode with unknown users, Hadoop employs negative caching so that if
the result of the lookup is empty, return an empty group directly instead of performing more
group mapping queries,
+The cache invalidation is configurable via `hadoop.security.groups.negative-cache.secs`.
The default is 30 seconds, so if group mapping service providers returns no group for a user,
no lookup will be performed for the same user within 30 seconds.
 
 LDAP Groups Mapping
 --------
@@ -85,9 +89,9 @@ in order to be considered a member.
 The default configuration supports LDAP group name resolution with an Active Directory server.
 
 ### POSIX Groups ###
-If the LDAP server supports POSIX group semantics, Hadoop can perform LDAP group resolution
queries to the server by setting both
-`hadoop.security.group.mapping.ldap.search.filter.user` to  `posixAccount` and
-`hadoop.security.group.mapping.ldap.search.filter.group` to `posixGroup`.
+If the LDAP server supports POSIX group semantics (RFC-2307), Hadoop can perform LDAP group
resolution queries to the server by setting both
+`hadoop.security.group.mapping.ldap.search.filter.user` to  `(&amp;(objectClass=posixAccount)(uid={0}))`
and
+`hadoop.security.group.mapping.ldap.search.filter.group` to `(objectClass=posixGroup)`.
 
 ### SSL ###
 To secure the connection, the implementation supports LDAP over SSL (LDAPS). SSL is enable
by setting `hadoop.security.group.mapping.ldap.ssl` to `true`.


Mime
View raw message