hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4336) YARN NodeManager - Container Initialization - Excessive load on NSS/LDAP
Date Fri, 06 Nov 2015 16:47:27 GMT

    [ https://issues.apache.org/jira/browse/YARN-4336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14993932#comment-14993932
] 

Jason Lowe commented on YARN-4336:
----------------------------------

bq. Seems like this could also be related...  https://issues.apache.org/jira/browse/HADOOP-12413
Nice find!  I totally missed that when it went by.  I'll pull that fix into the 2.6 and 2.7
lines.  I think that could eliminate the bogus lookups in practice when the reverse ACL isn't
being used.

bq.  Do you see an issue with my workaround for now in my own env until HWX can provide a
final solution?
It will work.  Nit: it's pricey to compile the pattern every time, could just compile it once.
 Or as I mentioned above, I think pulling in HADOOP-12413 to your build could also eliminate
the bogus lookups (assuming you don't use the reverse ACL feature).


> YARN NodeManager - Container Initialization - Excessive load on NSS/LDAP
> ------------------------------------------------------------------------
>
>                 Key: YARN-4336
>                 URL: https://issues.apache.org/jira/browse/YARN-4336
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.4.0, 2.4.1, 2.6.0, 2.7.0, 2.6.1, 2.7.1
>         Environment: NSS w/ SSSD or Dell/Quest - VASD
>            Reporter: Greg Senia
>            Assignee: Greg Senia
>         Attachments: YARN-4336-tactical.txt
>
>
> Hi folks after performing some debug for our Unix Engineering and Active Directory teams
it was discovered that on YARN Container Initialization a call via Hadoop Common AccessControlList.java:
>   for(String group: ugi.getGroupNames()) {
>         if (groups.contains(group)) {
>           return true;
>         }
>       }
> Unfortunately with the security call to check access on "appattempt_XXXXXXXXXXXXX_XXXXX_XXXXX"
will always return false but will make unnecessary calls to NameSwitch service on linux which
will call things like SSSD/Quest VASD which will then initiate LDAP calls looking for non
existent userid's causing excessive load on LDAP.
> For now our tactical work around is as follows:
> /**
>    * Checks if a user represented by the provided {@link UserGroupInformation}
>    * is a member of the Access Control List
>    * @param ugi UserGroupInformation to check if contained in the ACL
>    * @return true if ugi is member of the list
>    */
>   public final boolean isUserInList(UserGroupInformation ugi) {
>     if (allAllowed || users.contains(ugi.getShortUserName())) {
>       return true;
>     } else {
>         String patternString = "^appattempt_\\d+_\\d+_\\d+$";
>         Pattern pattern = Pattern.compile(patternString);
>         Matcher matcher = pattern.matcher(ugi.getShortUserName());
>         boolean matches = matcher.matches();
>         if (matches) {
>         	LOG.debug("Bailing !! AppAttempt Matches DONOT call UGI FOR GROUPS!!");;
>         	return false;
>         }
>     	
>     	
>       for(String group: ugi.getGroupNames()) {
>         if (groups.contains(group)) {
>           return true;
>         }
>       }
>     }
>     return false;
>   }
>   public boolean isUserAllowed(UserGroupInformation ugi) {
>     return isUserInList(ugi);
>   }
> Example of VASD Debug log showing the lookups for one task attempt 32 of them:
> One task:
> Oct 30 22:55:43 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:55:43 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:55:43 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:55:43 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:56:15 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:56:15 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:56:15 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:56:15 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:56:45 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:56:45 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:56:45 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:56:45 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:57:18 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:57:18 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:57:18 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:57:18 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:57:49 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:57:49 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:57:49 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:57:49 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:58:22 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:58:22 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:58:22 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:58:22 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:58:52 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:58:52 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:58:52 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:58:52 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:59:30 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:59:30 xhadoopm5d vasd[20741]: _vasug_user_namesearch_gc: searching GC for host
service domain EXNSD.EXA.EXAMPLE.COM with filter (&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))
> Oct 30 22:59:30 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>
> Oct 30 22:59:30 xhadoopm5d vasd[20741]: libvas_attrs_find_uri: Searching <GC://@EXNSD.EXA.EXAMPLE.COM>
with filter=<(&(objectCategory=Person)(samaccountname=appattempt_1446145939879_0022_000001))>,
base=<>, scope=<sub>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message