hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srikanth Upputuri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6606) Optimize HDFS Encrypted Transport performance
Date Wed, 17 Sep 2014 05:05:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136781#comment-14136781

Srikanth Upputuri commented on HDFS-6606:

{quote} In this JIRA, 3DES is used to encrypt/decrypt the negotiated cipher key (originally
it was used to encrypt the transferred data). You are right, the channel confidentiality is
the same, but it's enough. Our goal is to improve the performance.{quote}

Thank you for the explanation. I read about AES-NI and I now understand that with a JCE provider
like Diceros AES performance will significantly improve. However, if we need to provide support
for increased confidentiality with AES, can we not do it by implementing GSSAPI mechanism
in addition to the existing DIGEST-MD5, the same way it is implemented for rpc? The java gss
api has support for AES anyway as described in  http://docs.oracle.com/javase/7/docs/technotes/guides/security/jgss/jgss-features.html.
That way we get better performance (with AES-NI support) as well as better data privacy. I
have read through all the comments but didn't quite get why this approach is not considered.
Any reasons?

> Optimize HDFS Encrypted Transport performance
> ---------------------------------------------
>                 Key: HDFS-6606
>                 URL: https://issues.apache.org/jira/browse/HDFS-6606
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, hdfs-client, security
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>         Attachments: HDFS-6606.001.patch, HDFS-6606.002.patch, HDFS-6606.003.patch, HDFS-6606.004.patch,
> In HDFS-3637, [~atm] added support for encrypting the DataTransferProtocol, it was a
great work.
> It utilizes SASL {{Digest-MD5}} mechanism (use Qop: auth-conf),  it supports three security
> * high                      3des   or rc4 (128bits)
> * medium             des or rc4(56bits)
> * low                       rc4(40bits)
> 3des and rc4 are slow, only *tens of MB/s*, 
> http://www.javamex.com/tutorials/cryptography/ciphers.shtml
> http://www.cs.wustl.edu/~jain/cse567-06/ftp/encryption_perf/
> I will give more detailed performance data in future. Absolutely it’s bottleneck and
will vastly affect the end to end performance. 
> AES(Advanced Encryption Standard) is recommended as a replacement of DES, it’s more
secure; with AES-NI support, the throughput can reach nearly *2GB/s*, it won’t be the bottleneck
any more, AES and CryptoCodec work is supported in HADOOP-10150, HADOOP-10603 and HADOOP-10693
(We may need to add a new mode support for AES). 
> This JIRA will use AES with AES-NI support as encryption algorithm for DataTransferProtocol.

This message was sent by Atlassian JIRA

View raw message