From issues-return-95450-archive-asf-public=cust-asf.ponee.io@nifi.apache.org Thu Apr 9 03:24:03 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id E45C018057A for ; Thu, 9 Apr 2020 05:24:02 +0200 (CEST) Received: (qmail 34499 invoked by uid 500); 9 Apr 2020 03:24:02 -0000 Mailing-List: contact issues-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list issues@nifi.apache.org Received: (qmail 34485 invoked by uid 99); 9 Apr 2020 03:24:02 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 09 Apr 2020 03:24:02 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 358A4E0339 for ; Thu, 9 Apr 2020 03:24:01 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 6BBC278028F for ; Thu, 9 Apr 2020 03:24:00 +0000 (UTC) Date: Thu, 9 Apr 2020 03:24:00 +0000 (UTC) From: "Andy LoPresto (Jira)" To: issues@nifi.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (NIFI-7344) Flowfile content viewer appends extra bytes to cipher text MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/NIFI-7344?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-7344: -------------------------------- Description:=20 While investigating other issues with encryption services, I had serious di= fficulty decrypting some content that had been _encrypted_ in NiFi due to c= ipher pad block errors (almost always indicate the use of the wrong key or = the application of a key to the wrong cipher text) and block size errors (i= ndicate invalid cipher text length).=C2=A0 Through manual analysis, I found that when data is encrypted with {{Encrypt= Content}}, the data is encrypted correctly and the right bytes are written = to the content flowfile. In this case, using the following values, the outp= ut should always be 139 bytes. The flowfile size attribute indicates that c= orrectly, and downloading the flowfile content is exactly 139 bytes. Howeve= r, viewing the content in the embedded content viewer results in 140 bytes,= introducing a trailing {{0x10}} byte that should not be there.=C2=A0 So far I have only observed this using the {{Bcrypt}}=C2=A0KDF, and not usi= ng the {{PBKDF2}} KDF. I will continue to investigate if other KDFs influen= ce this outcome.=C2=A0 For plaintext or JSON/XML content, this is a trivial bug. However, when it = silently modifies binary cipher text, this is a critical, near blocker, iss= ue.=C2=A0 =C2=A0 *Plaintext*: \{{This is a plaintext message generated at ${now():format('YY= YY-MM-dd HH:mm:ss.SSS Z')}. }} -> Resolves to=C2=A0"This is a plaintext mes= sage generated at 2020-04-08 20:11:44.743 Z. " which is *68* bytes in UTF-8= .=C2=A0 *KDF*: Bcrypt *Encryption Method*: AES-CBC *Password*:=C2=A0thisIsABadPassword =C2=A0 The resulting cipher text should be 139 bytes (29 bytes of Bcrypt salt, 8 b= ytes of NiFi salt delimiter, 16 bytes of IV, 6 bytes of NiFi IV delimiter, = and 80 bytes of cipher text [see below for explanation]).=C2=A0 =C2=A0 This flow is not reproducible on {{master}} branch but can be done on NIFI-= 7122=C2=A0{{}}branch in my repo.=C2=A0=C2=A0 =C2=A0 Cipher text length calculation for AES-CBC: {code:java} int plaintextBlockLength =3D Math.ceil(plaintext.length() / 16.0) boolean plaintextIsExactBlockMultiple =3D plaintext.length() % 16 =3D=3D 0 int cipherTextLength =3D (plaintextIsExactBlockMultiple ? plaintextBlockLen= gth + 1 : plaintextBlockLength) * 16{code} Given the above formula, a plaintext of length 68 requires 5 blocks, but is= not an even multiple of 16, so does not require an additional padding bloc= k. Rather, 80 bytes (5 * 16) is used.=C2=A0 was: While investigating other issues with encryption services, I had serious di= fficulty decrypting some content that had been _encrypted_ in NiFi due to c= ipher pad block errors (almost always indicate the use of the wrong key or = the application of a key to the wrong cipher text) and block size errors (i= ndicate invalid cipher text length).=C2=A0 Through manual analysis, I found that when data is encrypted with {{Encrypt= Content}}, the data is encrypted correctly and the right bytes are written = to the content flowfile. In this case, using the following values, the outp= ut should always be 139 bytes. The flowfile size attribute indicates that c= orrectly, and downloading the flowfile content is exactly 139 bytes. Howeve= r, viewing the content in the embedded content viewer results in 140 bytes,= introducing a trailing {{0x10}} byte that should not be there.=C2=A0 So far I have only observed this using the {{Bcrypt}}=C2=A0KDF, and not usi= ng the {{PBKDF2}} KDF. I will continue to investigate if other KDFs influen= ce this outcome.=C2=A0 For plaintext or JSON/XML content, this is a trivial bug. However, when it = silently modifies binary cipher text, this is a critical, near blocker, iss= ue.=C2=A0 =C2=A0 *Plaintext*: \{{This is a plaintext message generated at ${now():format('YY= YY-MM-dd HH:mm:ss.SSS Z')}. }} -> Resolves to=C2=A0"This is a plaintext mes= sage generated at 2020-04-08 20:11:44.743 Z. " which is *68* bytes in UTF-8= .=C2=A0 *KDF*: Bcrypt *Encryption Method*: AES-CBC *Password*:=C2=A0thisIsABadPassword =C2=A0 The resulting cipher text should be 139 bytes (29 bytes of Bcrypt salt, 8 b= ytes of NiFi salt delimiter, 16 bytes of IV, 6 bytes of NiFi IV delimiter, = and 80 bytes of cipher text [see below for explanation]).=C2=A0 =C2=A0 This flow is not reproducible on {{master}} branch but can be done on {{NIF= I-7324}} branch in my repo.=C2=A0=C2=A0 =C2=A0 Cipher text length calculation for AES-CBC: {code:java} int plaintextBlockLength =3D Math.ceil(plaintext.length() / 16.0) boolean plaintextIsExactBlockMultiple =3D plaintext.length() % 16 =3D=3D 0 int cipherTextLength =3D (plaintextIsExactBlockMultiple ? plaintextBlockLen= gth + 1 : plaintextBlockLength) * 16{code} Given the above formula, a plaintext of length 68 requires 5 blocks, but is= not an even multiple of 16, so does not require an additional padding bloc= k. Rather, 80 bytes (5 * 16) is used.=C2=A0 > Flowfile content viewer appends extra bytes to cipher text > ---------------------------------------------------------- > > Key: NIFI-7344 > URL: https://issues.apache.org/jira/browse/NIFI-7344 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI, Security > Affects Versions: 1.12.0, 1.11.4 > Reporter: Andy LoPresto > Priority: Critical > Labels: bcrypt, content-viewer, encryption, kdf, security > Attachments: Screen Shot 2020-04-08 at 7.54.59 PM.png, Screen Sho= t 2020-04-08 at 8.19.06 PM.png, Screen Shot 2020-04-08 at 8.20.02 PM.png, f= low_20200408-201821_NIFI-7344.xml.gz > > > While investigating other issues with encryption services, I had serious = difficulty decrypting some content that had been _encrypted_ in NiFi due to= cipher pad block errors (almost always indicate the use of the wrong key o= r the application of a key to the wrong cipher text) and block size errors = (indicate invalid cipher text length).=C2=A0 > Through manual analysis, I found that when data is encrypted with {{Encry= ptContent}}, the data is encrypted correctly and the right bytes are writte= n to the content flowfile. In this case, using the following values, the ou= tput should always be 139 bytes. The flowfile size attribute indicates that= correctly, and downloading the flowfile content is exactly 139 bytes. Howe= ver, viewing the content in the embedded content viewer results in 140 byte= s, introducing a trailing {{0x10}} byte that should not be there.=C2=A0 > So far I have only observed this using the {{Bcrypt}}=C2=A0KDF, and not u= sing the {{PBKDF2}} KDF. I will continue to investigate if other KDFs influ= ence this outcome.=C2=A0 > For plaintext or JSON/XML content, this is a trivial bug. However, when i= t silently modifies binary cipher text, this is a critical, near blocker, i= ssue.=C2=A0 > =C2=A0 > *Plaintext*: \{{This is a plaintext message generated at ${now():format('= YYYY-MM-dd HH:mm:ss.SSS Z')}. }} -> Resolves to=C2=A0"This is a plaintext m= essage generated at 2020-04-08 20:11:44.743 Z. " which is *68* bytes in UTF= -8.=C2=A0 > *KDF*: Bcrypt > *Encryption Method*: AES-CBC > *Password*:=C2=A0thisIsABadPassword > =C2=A0 > The resulting cipher text should be 139 bytes (29 bytes of Bcrypt salt, 8= bytes of NiFi salt delimiter, 16 bytes of IV, 6 bytes of NiFi IV delimiter= , and 80 bytes of cipher text [see below for explanation]).=C2=A0 > =C2=A0 > This flow is not reproducible on {{master}} branch but can be done on NIF= I-7122=C2=A0{{}}branch in my repo.=C2=A0=C2=A0 > =C2=A0 > Cipher text length calculation for AES-CBC: > {code:java} > int plaintextBlockLength =3D Math.ceil(plaintext.length() / 16.0) > boolean plaintextIsExactBlockMultiple =3D plaintext.length() % 16 =3D=3D = 0 > int cipherTextLength =3D (plaintextIsExactBlockMultiple ? plaintextBlockL= ength + 1 : plaintextBlockLength) * 16{code} > Given the above formula, a plaintext of length 68 requires 5 blocks, but = is not an even multiple of 16, so does not require an additional padding bl= ock. Rather, 80 bytes (5 * 16) is used.=C2=A0 -- This message was sent by Atlassian Jira (v8.3.4#803005)