hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-5010) Reduce the frequency of getCurrentUser() calls from namenode
Date Thu, 18 Jul 2013 23:04:48 GMT

     [ https://issues.apache.org/jira/browse/HDFS-5010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kihwal Lee updated HDFS-5010:
-----------------------------

    Description: 
According to the "worst" jstack of a busy namenode I took, there were 29 BLOCKED handler threads.
All of them were blocked on the UGI class lock. Here is the breakdown:
- 2 ensureInitialized() - from non static synchronized methods. This Jira will unblock these.
- 27 getCurrentUser()

Among the 27 threads that were blocked at getCurrentUser(),
- 18 FSPermissionChecker() - from FSNamesystem#getPermissionChecker() in most namenode RPC
methods
- 8 BlockTokenSecretManager#generateToken() - getBlockLocations()
- 1 NameNodeRpcServer.mkdirs

I think FSPermissionChecker can be modified to be created with a passed in UGI. FSNamesystem
can the one already stored in RPC server by calling getRemoteUser(). This will eliminate a
bulk of getCurrentUser() calls from namenode RPC handlers. A similar change can be made to
mkdirs. Block token generation is not as straightforward. Even without it we can eliminate
majority of the calls.

  was:
According to the "worst" jstack of a busy namenode I took, there were 29 BLOCKED handler threads.
All of them were blocked on the UGI class lock. Here is the breakdown:
-2 ensureInitialized() - from non static synchronized methods. This Jira will unblock these.
-27 getCurrentUser()

Among the 27 threads that were blocked at getCurrentUser(),
- 18 FSPermissionChecker() - from FSNamesystem#getPermissionChecker() in most namenode RPC
methods
- 8 BlockTokenSecretManager#generateToken() - getBlockLocations()
- 1 NameNodeRpcServer.mkdirs

I think FSPermissionChecker can be modified to be created with a passed in UGI. FSNamesystem
can the one already stored in RPC server by calling getRemoteUser(). This will eliminate a
bulk of getCurrentUser() calls from namenode RPC handlers. A similar change can be made to
mkdirs. Block token generation is not as straightforward. Even without it we can eliminate
majority of the calls.

    
> Reduce the frequency of getCurrentUser() calls from namenode
> ------------------------------------------------------------
>
>                 Key: HDFS-5010
>                 URL: https://issues.apache.org/jira/browse/HDFS-5010
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Kihwal Lee
>
> According to the "worst" jstack of a busy namenode I took, there were 29 BLOCKED handler
threads. All of them were blocked on the UGI class lock. Here is the breakdown:
> - 2 ensureInitialized() - from non static synchronized methods. This Jira will unblock
these.
> - 27 getCurrentUser()
> Among the 27 threads that were blocked at getCurrentUser(),
> - 18 FSPermissionChecker() - from FSNamesystem#getPermissionChecker() in most namenode
RPC methods
> - 8 BlockTokenSecretManager#generateToken() - getBlockLocations()
> - 1 NameNodeRpcServer.mkdirs
> I think FSPermissionChecker can be modified to be created with a passed in UGI. FSNamesystem
can the one already stored in RPC server by calling getRemoteUser(). This will eliminate a
bulk of getCurrentUser() calls from namenode RPC handlers. A similar change can be made to
mkdirs. Block token generation is not as straightforward. Even without it we can eliminate
majority of the calls.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message