hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10768) Optimize Hadoop RPC encryption performance
Date Thu, 14 Apr 2016 12:34:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241050#comment-15241050

Kai Zheng commented on HADOOP-10768:

Thanks [~dian.fu] for working on and attacking this!
I only did a quick look at the work. So far some questions in high level:
* Would you have a design doc that describes the requirement, the approach? I understand this
was well discussed in the past, but guess a doc like this may be good to summarize and bring
fresh discussion.
* I guess it's all about and for performance. Do you have any number to share?
* What's the impact? Does it mean to upgrade RPC version? Can external clients still be able
to talk with the server via SASL? How this affect downstream components?
* Looks like the work is mainly in SASL layer, when Kerberos is enabled, will it still favor
the GSSAPI mechanism? If not or it's bypassed, what encryption key is used and how it's obtained?
* The patch looks rather large, the change covering crypto, protocol, sasl rpc client and
server, data transfer and some misc. Would you break it down? This one can be the umbrella.

Thanks again!

> Optimize Hadoop RPC encryption performance
> ------------------------------------------
>                 Key: HADOOP-10768
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10768
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: performance, security
>    Affects Versions: 3.0.0
>            Reporter: Yi Liu
>            Assignee: Dian Fu
>         Attachments: HADOOP-10768.001.patch, HADOOP-10768.002.patch
> Hadoop RPC encryption is enabled by setting {{hadoop.rpc.protection}} to "privacy". It
utilized SASL {{GSSAPI}} and {{DIGEST-MD5}} mechanisms for secure authentication and data
protection. Even {{GSSAPI}} supports using AES, but without AES-NI support by default, so
the encryption is slow and will become bottleneck.
> After discuss with [~atm], [~tucu00] and [~umamaheswararao], we can do the same optimization
as in HDFS-6606. Use AES-NI with more than *20x* speedup.
> On the other hand, RPC message is small, but RPC is frequent and there may be lots of
RPC calls in one connection, we needs to setup benchmark to see real improvement and then
make a trade-off. 

This message was sent by Atlassian JIRA

View raw message