nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Moser (JIRA)" <j...@apache.org>
Subject [jira] [Created] (NIFI-2787) PersistentProvenanceRepository rollover can fail on immense indexed attributes
Date Fri, 16 Sep 2016 21:52:20 GMT
Michael Moser created NIFI-2787:
-----------------------------------

             Summary: PersistentProvenanceRepository rollover can fail on immense indexed
attributes
                 Key: NIFI-2787
                 URL: https://issues.apache.org/jira/browse/NIFI-2787
             Project: Apache NiFi
          Issue Type: Bug
          Components: Core Framework
    Affects Versions: 0.7.0, 1.0.0
            Reporter: Michael Moser


Accidentally created an immense attribute (36,000 bytes), which I indexed with nifi.provenance.repository.indexed.attributes.
 Received this error.

ERROR [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Failed
to rollover Provenance repository due to java.lang.IllegalArgumentException: Document contains
at least one immense term in field="FOO" (whose UTF8 encoding is longer than the max length
32766), all of which were skipped. Please correct the analyzer to not produce such terms.

Perhaps this is as simple as changing https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/RepositoryConfiguration.java#L37
to 32766 to match Lucene.  Investigation & testing needed.

For background, this Lucene ticket made exceeding the term size limit an IllegalArgumentException
https://issues.apache.org/jira/browse/LUCENE-5472




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message