From common-issues-return-152441-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Mon May 14 17:31:09 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 5084118076D for ; Mon, 14 May 2018 17:31:09 +0200 (CEST) Received: (qmail 70977 invoked by uid 500); 14 May 2018 15:31:08 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 70959 invoked by uid 99); 14 May 2018 15:31:08 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 May 2018 15:31:08 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id A2A9CC665B for ; Mon, 14 May 2018 15:31:07 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 2H2l2uhJ5cBL for ; Mon, 14 May 2018 15:31:07 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 53C885FB6A for ; Mon, 14 May 2018 15:31:06 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 8B042E0110 for ; Mon, 14 May 2018 15:31:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 4D20E21298 for ; Mon, 14 May 2018 15:31:00 +0000 (UTC) Date: Mon, 14 May 2018 15:31:00 +0000 (UTC) From: "Wei-Chiu Chuang (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HADOOP-10768) Optimize Hadoop RPC encryption performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-10768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16474362#comment-16474362 ] Wei-Chiu Chuang commented on HADOOP-10768: ------------------------------------------ Rev08 test passed our internal smoke tests after HBASE-20572 is applied. I'll spend some time today to review it. Thanks. > Optimize Hadoop RPC encryption performance > ------------------------------------------ > > Key: HADOOP-10768 > URL: https://issues.apache.org/jira/browse/HADOOP-10768 > Project: Hadoop Common > Issue Type: Improvement > Components: performance, security > Affects Versions: 3.0.0-alpha1 > Reporter: Yi Liu > Assignee: Dapeng Sun > Priority: Major > Attachments: HADOOP-10768.001.patch, HADOOP-10768.002.patch, HADOOP-10768.003.patch, HADOOP-10768.004.patch, HADOOP-10768.005.patch, HADOOP-10768.006.patch, HADOOP-10768.007.patch, HADOOP-10768.008.patch, HADOOP-10768.009.patch, Optimize Hadoop RPC encryption performance.pdf > > > Hadoop RPC encryption is enabled by setting {{hadoop.rpc.protection}} to "privacy". It utilized SASL {{GSSAPI}} and {{DIGEST-MD5}} mechanisms for secure authentication and data protection. Even {{GSSAPI}} supports using AES, but without AES-NI support by default, so the encryption is slow and will become bottleneck. > After discuss with [~atm], [~tucu00] and [~umamaheswararao], we can do the same optimization as in HDFS-6606. Use AES-NI with more than *20x* speedup. > On the other hand, RPC message is small, but RPC is frequent and there may be lots of RPC calls in one connection, we needs to setup benchmark to see real improvement and then make a trade-off. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: common-issues-help@hadoop.apache.org