nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mengze Li (Jira)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-7856) Provenance failed to be compressed after nifi upgrade to 1.12
Date Fri, 02 Oct 2020 15:06:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-7856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206244#comment-17206244
] 

Mengze Li commented on NIFI-7856:
---------------------------------

Hi [~markap14], that's a great question since that's one major reason that we upgraded to
1.12.0 since sometimes the data provenance is missing in the processor view but visible in
the global view.
So to answer your question, we do both and in 1.12.0, the missing and delaying data provenance
records are consistent from either view.
In global view, we usually get component id and search that way since it is unique.
We are speculating that some processes is compressing the files before the scheduled time
and those ones didn't make it to the lucene index to be searched.
Will try to dig around the logs more to provide information that I can could help you debug
further.

> Provenance failed to be compressed after nifi upgrade to 1.12
> -------------------------------------------------------------
>
>                 Key: NIFI-7856
>                 URL: https://issues.apache.org/jira/browse/NIFI-7856
>             Project: Apache NiFi
>          Issue Type: Bug
>    Affects Versions: 1.12.0
>            Reporter: Mengze Li
>            Priority: Major
>         Attachments: 1683472.prov, ls.png, screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> We upgraded our nifi cluster from 1.11.3 to 1.12.0.
> The nodes come up and everything looks to be functional. I can see 1.12.0 is running.
> Later on, we discovered that the data provenance is missing. From checking our logs,
we see tons of errors compressing the logs.
> {code}
> 2020-09-28 03:38:35,205 ERROR [Compress Provenance Logs-1-thread-1] o.a.n.p.s.EventFileCompressor
Failed to compress ./provenance_repository/2752821.prov on rollover
> {code}
> This didn't happen in 1.11.3. 
> Is this a known issue? We are considering reverting back if there is no solution for
this since we can't go prod with no/broken data provenance.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message