hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9937) Improvement of the group information refer frequency
Date Mon, 09 Sep 2013 17:23:52 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13762056#comment-13762056
] 

Colin Patrick McCabe commented on HADOOP-9937:
----------------------------------------------

We put a lock around getgrname_r by default, since a large percentage of systems out there
have an implementation that actually is not re-entrant (despite the _r).  So you may not get
as large a speedup as you think by removing the lock in {{UserGroupInformation#getGroupNames}}
(unless you also set {{hadoop.workaround.non.threadsafe.getpwuid}} to {{false}}).

You could create a custom lock just for group information (rather than synchronizing on the
big UGI lock), but I'm not sure how much that would help.  Do you have a more detailed proposal
for how to change the synchronization here?
                
> Improvement of the group information refer frequency
> ----------------------------------------------------
>
>                 Key: HADOOP-9937
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9937
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: security
>    Affects Versions: 3.0.0
>            Reporter: Shinichi Yamashita
>            Priority: Minor
>         Attachments: namenode-threaddump.txt
>
>
> The node (e.g. NameNode, ResourceManager) uses UGI.getGroupNames() now to get the information
of the user's group who accessed it.
> In UGI.getGroupsNames(), synchronized is declared, but UGI instance by various methods
each time and the node get different lock in getGroupName().
> For example, when cache time limit in userToGroupsMap of Groups expired and the node
accepts many requests at the same time, each refers for group information in id command or
JNI.
> To improve the refer frequency of the group information, it should change about synchronization.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message