Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Mon, 27 Oct 2014 22:46:34 +0000 (UTC)
From: "Chris Nauroth (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12724375.1404097994000.348435.1414449994848@Atlassian.JIRA>
In-Reply-To: <JIRA.12724375.1404097994000@Atlassian.JIRA>
References: <JIRA.12724375.1404097994000@Atlassian.JIRA>
 <JIRA.12724375.1404097994529@arcas>
Subject: [jira] [Commented] (HDFS-6606) Optimize HDFS Encrypted Transport
 performance
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HDFS-6606?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D14185=
955#comment-14185955 ]=20

Chris Nauroth commented on HDFS-6606:
-------------------------------------

Thanks for checking in with your final review, Aaron.

Yi, I reviewed the latest patch once again, and I found one more potential =
issue in {{SaslDataTransferServer}}:

{code}
      CipherOption cipherOption =3D null;
      if (sasl.isNegotiatedQopPrivacy()) {
        // Negotiate a cipher option
        cipherOption =3D negotiateCipherOption(dnConf.getConf(), cipherOpti=
ons);
        if (LOG.isDebugEnabled()) {
          LOG.debug("Server using cipher suite " +=20
              cipherOption.getCipherSuite().getName());
        }
      }
{code}

It's possible for {{negotiateCipherOption}} to return {{null}} when the con=
nection comes from an older client version that doesn't do cipher negotiati=
on.  If debug logging is enabled, then the log statement would cause a {{Nu=
llPointerException}}.

I'll be +1 after that's addressed, and I'm happy to volunteer for the commi=
t.

> Optimize HDFS Encrypted Transport performance
> ---------------------------------------------
>
>                 Key: HDFS-6606
>                 URL: https://issues.apache.org/jira/browse/HDFS-6606
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, hdfs-client, security
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>         Attachments: HDFS-6606.001.patch, HDFS-6606.002.patch, HDFS-6606.=
003.patch, HDFS-6606.004.patch, HDFS-6606.005.patch, HDFS-6606.006.patch, H=
DFS-6606.007.patch, HDFS-6606.008.patch, OptimizeHdfsEncryptedTransportperf=
ormance.pdf
>
>
> In HDFS-3637, [~atm] added support for encrypting the DataTransferProtoco=
l, it was a great work.
> It utilizes SASL {{Digest-MD5}} mechanism (use Qop: auth-conf),  it suppo=
rts three security strength:
> * high                      3des   or rc4 (128bits)
> * medium             des or rc4(56bits)
> * low                       rc4(40bits)
> 3des and rc4 are slow, only *tens of MB/s*,=20
> http://www.javamex.com/tutorials/cryptography/ciphers.shtml
> http://www.cs.wustl.edu/~jain/cse567-06/ftp/encryption_perf/
> I will give more detailed performance data in future. Absolutely it=E2=80=
=99s bottleneck and will vastly affect the end to end performance.=20
> AES(Advanced Encryption Standard) is recommended as a replacement of DES,=
 it=E2=80=99s more secure; with AES-NI support, the throughput can reach ne=
arly *2GB/s*, it won=E2=80=99t be the bottleneck any more, AES and CryptoCo=
dec work is supported in HADOOP-10150, HADOOP-10603 and HADOOP-10693 (We ma=
y need to add a new mode support for AES).=20
> This JIRA will use AES with AES-NI support as encryption algorithm for Da=
taTransferProtocol.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)