hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srikanth Upputuri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6606) Optimize HDFS Encrypted Transport performance
Date Fri, 12 Sep 2014 09:04:36 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131300#comment-14131300
] 

Srikanth Upputuri commented on HDFS-6606:
-----------------------------------------

This is a very nice effort. It's a great deal of learning for me reading through this jira
and HDFS-3637. But I have a couple of fundamental questions here.

Does this patch improve data transfer speed? But isn't the existing RC4 option much faster
(as shown in the comparison analysis)?

Does this patch improve the data transfer channel confidentiality? But, if we transfer the
AES keys and IVs over a 3DES encrypted channel, isn't the overall confidentiality effectively
same as someone who can successfully intercept and decrypt the 3DES traffic can read the AES
keys?

Am I missing something here?

> Optimize HDFS Encrypted Transport performance
> ---------------------------------------------
>
>                 Key: HDFS-6606
>                 URL: https://issues.apache.org/jira/browse/HDFS-6606
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, hdfs-client, security
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>         Attachments: HDFS-6606.001.patch, HDFS-6606.002.patch, HDFS-6606.003.patch, HDFS-6606.004.patch,
OptimizeHdfsEncryptedTransportperformance.pdf
>
>
> In HDFS-3637, [~atm] added support for encrypting the DataTransferProtocol, it was a
great work.
> It utilizes SASL {{Digest-MD5}} mechanism (use Qop: auth-conf),  it supports three security
strength:
> * high                      3des   or rc4 (128bits)
> * medium             des or rc4(56bits)
> * low                       rc4(40bits)
> 3des and rc4 are slow, only *tens of MB/s*, 
> http://www.javamex.com/tutorials/cryptography/ciphers.shtml
> http://www.cs.wustl.edu/~jain/cse567-06/ftp/encryption_perf/
> I will give more detailed performance data in future. Absolutely it’s bottleneck and
will vastly affect the end to end performance. 
> AES(Advanced Encryption Standard) is recommended as a replacement of DES, it’s more
secure; with AES-NI support, the throughput can reach nearly *2GB/s*, it won’t be the bottleneck
any more, AES and CryptoCodec work is supported in HADOOP-10150, HADOOP-10603 and HADOOP-10693
(We may need to add a new mode support for AES). 
> This JIRA will use AES with AES-NI support as encryption algorithm for DataTransferProtocol.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message