hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1043) Benchmark overhead of server-side group resolution of users
Date Fri, 19 Mar 2010 19:20:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847522#action_12847522

Konstantin Shvachko commented on HDFS-1043:

I ran the benchmark on three versions of hadoop 
# 0.20.1, which does not have any security code, and therefore kerberos and delegation token
authentications are not applicable there.
# 0.20.100, which contains the latest state of security implementation
# 0.22.trunk, which does not have all the latest security patches applied at the time of benchmarking
(just for the reference)

The benchmark creates a connection to the RPC server 1000 times. Each time the RPC server
authenticates the client using one of the three authentication methods (no authentication,
kerberos, delegation token). The result if the average latency of the connection request.

The table below shows that 
- when security is turned off the the new code still adds 14% overhead.
- The overhead for kerberos authentication is predictably huge.
- The delegation token authentication was intended as a fast alternative to kerberos. It is
somewhat faster, but not as nearly as the non-secure version. This should definitely be the
focus of future optimizations.
- 0.22 is 1-2% slower compared to 0.20.100. It is expected to catch up with it, when all latest
security contributions are ported to the trunk.

||Version||No security||Kerberos||Delegation Tooken||
|0.20.1  |0.920| | |
|0.20.100|1.047 (+14%)|44.670|42.615|
|0.22    |1.597 (+73%)|45.148|43.455|

> Benchmark overhead of server-side group resolution of users
> -----------------------------------------------------------
>                 Key: HDFS-1043
>                 URL: https://issues.apache.org/jira/browse/HDFS-1043
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: benchmarks
>    Affects Versions: 0.22.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>             Fix For: 0.22.0
>         Attachments: UGCRefresh.patch
> Server-side user group resolution was introduced in HADOOP-4656. 
> The benchmark should repeatedly request the name-node for user group resolution, and
reset NN's user group cache periodically.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message