hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4532) RPC call queue may fill due to current user lookup
Date Fri, 01 Mar 2013 16:01:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590656#comment-13590656
] 

Kihwal Lee commented on HDFS-4532:
----------------------------------

The patch itself looks good, but there is one thing I am not sure about in the existing code.
Do isExternalInvocation() and FSNamesystem.getRemoteIp() work correctly when Hftp is used?
                
> RPC call queue may fill due to current user lookup
> --------------------------------------------------
>
>                 Key: HDFS-4532
>                 URL: https://issues.apache.org/jira/browse/HDFS-4532
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
>         Attachments: HDFS-4532.branch-23.patch, HDFS-4532.patch
>
>
> Audit logging need to invoke {{UGI.getCurrentUser}} to log the user performing an operation.
 This is a synch'ed method, so audit logging is effectively serializing the completion of
concurrent read ops.  Under very heavy load, RPC throughput drops dramatically, the call queue
fills with thousands of waiting operations, and clients begin timing out.  Jstack shows all
but one call handler blocked on {{UGI.getCurrentUser}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message