From commits-return-66147-archive-asf-public=cust-asf.ponee.io@pulsar.apache.org Tue Nov 3 14:18:28 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-he-de.apache.org (mxout1-he-de.apache.org [95.216.194.37]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 9A5AC18064E for ; Tue, 3 Nov 2020 15:18:28 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-he-de.apache.org (ASF Mail Server at mxout1-he-de.apache.org) with SMTP id 1D38364254 for ; Tue, 3 Nov 2020 14:18:28 +0000 (UTC) Received: (qmail 57833 invoked by uid 500); 3 Nov 2020 14:18:27 -0000 Mailing-List: contact commits-help@pulsar.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pulsar.apache.org Delivered-To: mailing list commits@pulsar.apache.org Received: (qmail 57822 invoked by uid 99); 3 Nov 2020 14:18:27 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Nov 2020 14:18:27 +0000 From: =?utf-8?q?GitBox?= To: commits@pulsar.apache.org Subject: =?utf-8?q?=5BGitHub=5D_=5Bpulsar=5D_nlu90_opened_a_new_pull_request_=238432?= =?utf-8?q?=3A_=5BIssue_8382=5D=5BPulsar_Function=5D_Enable_e2e_encryption_f?= =?utf-8?q?or_Pulsar_Function?= Message-ID: Date: Tue, 03 Nov 2020 14:18:27 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit nlu90 opened a new pull request #8432: URL: https://github.com/apache/pulsar/pull/8432 *(If this PR fixes a github issue, please add `Fixes #`.)* Fixes #8382 ### Motivation Add the e2e encryption support for Pulsar Functions ### Modifications - Add `CryptoConfig` the encapsulate all the crypto related configs set by user - Add `CryptoSpec` to `Function` protobuf to container crypto information internally - Add `CryptoUtils` to help create instance, convert between `CryptoConfig` and `CryptoSpec` - Add crypto validation method in `ValidatorUtils` to ensure the provided `CryptoKeyReader` Class has a ctor with `Map` arg - Updated the cli to allow user set crypto for consumer/producer when submitting the function - Update `PulsarSource`, `PulsarSink` to use the crypto config if provided ### Verifying this change - [ ] Make sure that the change passes the CI checks. This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): Added `pulsar-client-messagecrypto-bc` into `puslar-function instance` - The admin cli options: Added `--producer-config` to allow setting crypto related configs. ### Documentation - Does this pull request introduce a new feature? Yes - If yes, how is the feature documented? not documented - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation: [#8431](https://github.com/apache/pulsar/issues/8431) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org