hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4353) Provide short circuit user group mapping for NM/AM
Date Thu, 28 Jan 2016 00:41:40 GMT

    [ https://issues.apache.org/jira/browse/YARN-4353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15120518#comment-15120518
] 

Daniel Templeton commented on YARN-4353:
----------------------------------------

The base issue here is that in a secure LDAP environment, the NM wants credentials to resolve
the groups even though it doesn't use them.  The issue there is that the AM shares the config
with the NM, meaning that exposing the credentials to the NM opens them up to the users.

[~vinodkv] said earlier:

bq. Regarding this JIRA, a lots of places in NM do depend on the user/group information

I looked, but I couldn't find them.  I even instrumented the {{UserGroupInformation.getGroupNames()}}
method and ran some different kinds of jobs: nothing.  Outside of code to facilitate testing,
{{UserGroupInformation.getGroupNames()}} is the only thing that uses the groups data in the
NM, MR AM, or dist shell AM.  What am I missing?

Grabbing the user group data and localizing it for the AM is a really nice idea, but given
that no one is using the data, it seems like extra work for no present reason.  Given that
the existence of the {{NullGroupMapping}} (HADOOP-12566) provides a clean way to avoid sharing
the credentials, however, I'm happy to convert this JIRA over to storing the groups data for
the AM that may one day eventually need it.

> Provide short circuit user group mapping for NM/AM
> --------------------------------------------------
>
>                 Key: YARN-4353
>                 URL: https://issues.apache.org/jira/browse/YARN-4353
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 2.7.1
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
>         Attachments: YARN-4353.prelim.patch
>
>
> When the NM launches an AM, the {{ContainerLocalizer}} gets the current user from {{UserGroupInformation}},
which triggers user group mapping, even though the user groups are never accessed.  If secure
LDAP is configured for group mapping, then there are some additional complications created
by the unnecessary group resolution.  Additionally, it adds unnecessary latency to the container
launch time.
> To address the issue, before getting the current user, the {{ContainerLocalizer}} should
configure {{UserGroupInformation}} with a null group mapping service that quickly and quietly
returns an empty group list for all users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message