Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 0D1FA200B82 for ; Fri, 16 Sep 2016 23:52:22 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 0B164160ADB; Fri, 16 Sep 2016 21:52:22 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 71BC5160AB7 for ; Fri, 16 Sep 2016 23:52:21 +0200 (CEST) Received: (qmail 52826 invoked by uid 500); 16 Sep 2016 21:52:20 -0000 Mailing-List: contact issues-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list issues@nifi.apache.org Received: (qmail 52810 invoked by uid 99); 16 Sep 2016 21:52:20 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 16 Sep 2016 21:52:20 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 77EE02C014C for ; Fri, 16 Sep 2016 21:52:20 +0000 (UTC) Date: Fri, 16 Sep 2016 21:52:20 +0000 (UTC) From: "Michael Moser (JIRA)" To: issues@nifi.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (NIFI-2787) PersistentProvenanceRepository rollover can fail on immense indexed attributes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Fri, 16 Sep 2016 21:52:22 -0000 Michael Moser created NIFI-2787: ----------------------------------- Summary: PersistentProvenanceRepository rollover can fail on immense indexed attributes Key: NIFI-2787 URL: https://issues.apache.org/jira/browse/NIFI-2787 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.7.0, 1.0.0 Reporter: Michael Moser Accidentally created an immense attribute (36,000 bytes), which I indexed with nifi.provenance.repository.indexed.attributes. Received this error. ERROR [Provenance Repository Rollover Thread-1] o.a.n.p.PersistentProvenanceRepository Failed to rollover Provenance repository due to java.lang.IllegalArgumentException: Document contains at least one immense term in field="FOO" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. Perhaps this is as simple as changing https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/RepositoryConfiguration.java#L37 to 32766 to match Lucene. Investigation & testing needed. For background, this Lucene ticket made exceeding the term size limit an IllegalArgumentException https://issues.apache.org/jira/browse/LUCENE-5472 -- This message was sent by Atlassian JIRA (v6.3.4#6332)