Mailing-List: contact issues-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@nifi.apache.org
Date: Mon, 6 Feb 2017 17:52:41 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@nifi.apache.org
Message-ID: <JIRA.13021519.1479402884000.7731.1486403561816@Atlassian.JIRA>
In-Reply-To: <JIRA.13021519.1479402884000@Atlassian.JIRA>
References: <JIRA.13021519.1479402884000@Atlassian.JIRA> <JIRA.13021519.1479402884825@jira-lw-us.apache.org>
Subject: [jira] [Commented] (NIFI-3055) StandardRecordWriter can throw
 UTFDataFormatException
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 06 Feb 2017 17:53:03 -0000


    [ https://issues.apache.org/jira/browse/NIFI-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854445#comment-15854445 ] 

ASF GitHub Bot commented on NIFI-3055:
--------------------------------------

Github user jskora commented on the issue:

    https://github.com/apache/nifi/pull/1470
  
    Closing this PR, committed on [4f72e34](https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=4f72e3491f2372c8c45afb96a765c1f5cdd2f07d).


> StandardRecordWriter can throw UTFDataFormatException
> -----------------------------------------------------
>
>                 Key: NIFI-3055
>                 URL: https://issues.apache.org/jira/browse/NIFI-3055
>             Project: Apache NiFi
>          Issue Type: Bug
>    Affects Versions: 1.0.0, 0.7.1
>            Reporter: Brandon DeVries
>            Assignee: Joe Skora
>            Priority: Blocker
>             Fix For: 0.8.0, 1.2.0
>
>
> StandardRecordWriter.writeRecord()\[1] uses DataOutputStream.writeUTF()\[2] without checking the length of the value to be written.  If this length is greater than  65535 (2^16 - 1), you get a UTFDataFormatException "encoded string too long..."\[3].  Ultimately, this can result in an IllegalStateException\[4], -bringing a halt to the data flow- causing PersistentProvenanceRepository "Unable to merge <prov_journal> with other Journal Files due to..." WARNings.
> Several of the field values being written in this way are pre-defined, and thus not likely an issue.  However, the "details" field can be populated by a processor, and can be of an arbitrary length.  -Additionally, if the detail filed is indexed (which I didn't investigate, but I'm sure is easy enough to determine), then the length might be subject to the Lucene limit discussed in NIFI-2787-.
> \[1] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/StandardRecordWriter.java#L163-L173
> \[2] http://docs.oracle.com/javase/7/docs/api/java/io/DataOutputStream.html#writeUTF%28java.lang.String%29
> \[3] http://stackoverflow.com/questions/22741556/dataoutputstream-purpose-of-the-encoded-string-too-long-restriction
> \[4] https://github.com/apache/nifi/blob/5fd4a55791da27fdba577636ac985a294618328a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/PersistentProvenanceRepository.java#L754-L755


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)