nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Firenz (Jira)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-6777) Data Provenance SIGSEGV : Search Lucene Index-1
Date Wed, 16 Oct 2019 17:24:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16953028#comment-16953028
] 

Firenz commented on NIFI-6777:
------------------------------

Thanks for you answer. We are using G1GC. So, we've just disabled it on our staging, and we'll
rollout on production next week. I'll let you know the result in 1 month, or the next crash
if any.

> Data Provenance SIGSEGV : Search Lucene Index-1
> -----------------------------------------------
>
>                 Key: NIFI-6777
>                 URL: https://issues.apache.org/jira/browse/NIFI-6777
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 1.8.0
>         Environment: Linux CentOS Linux release 7.5.1804 (Core)
> Linux 3.10.0-862.11.6.el7.x86_64
> jre1.8.0_181-amd64 et jre1.8.0_221-amd64
>            Reporter: Firenz
>            Priority: Major
>
> NIFI crashes with a core dump when using data provenance (5 times in 1 months, 4 differents
VMs).
>  
> It happens radomly when one of my data analyst is using dataprovenance, but the thread
is always "Search Lucene Index-1". My monitoring does not show any memory or thread issues
(overload...)
>  
> Any advice ? (I cannot attach the hs_err_pid54305.log, token issue with Jira ?)
>  
> Beginning of core dump : 
> {code:java}
> # JRE version: Java(TM) SE Runtime Environment (8.0_181-b13) (build 1.8.0_181-b13)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode linux-amd64 compressed
oops)
> # Problematic frame:
> # J 33233 C2 org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock()V (448
bytes) @ 0x00007f9c52baabf9 [0x00007f9
> c52baaa40+0x1b9]
> #
> # Core dump written. Default location: /appli/nifi-1.8.0/core or core.54305
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #---------------  T H R E A D  ---------------Current thread (0x00007f9c3c02b000):  JavaThread
"Search Lucene Index-1" daemon [_thread_in_Java, id=64766, stack(0x00007f9b9a
> c4d000,0x00007f9b9ad4e000)]siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR),
si_addr: 0x00007f9b82019d0c
> {code}
> My Dataprovenance config :
> {code:java}
> # Persistent Provenance Repository Properties
> nifi.provenance.repository.directory.default=/data_nifi_provenance_repo/1.8.0
> nifi.provenance.repository.max.storage.time=480 hours
> nifi.provenance.repository.max.storage.size=9 GB
> nifi.provenance.repository.rollover.time=30 secs
> nifi.provenance.repository.rollover.size=100 MB
> nifi.provenance.repository.query.threads=2
> nifi.provenance.repository.index.threads=2
> nifi.provenance.repository.compress.on.rollover=true
> nifi.provenance.repository.always.sync=false
> nifi.provenance.repository.journal.count=16
> # Comma-separated list of fields. Fields that are not indexed will not be searchable.
Valid fields are:
> # EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI,
Relationship, Details
> nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID,
Relationship
> # FlowFile Attributes that should be indexed and made searchable.  Some examples to consider
are filename, uuid, mime.type
> nifi.provenance.repository.indexed.attributes=external_id, batch_id, property_id, property_code,
http.headers.x-request-id
> # Large values for the shard size will result in more Java heap usage when searching
the Provenance Repository
> # but should provide better performance
> nifi.provenance.repository.index.shard.size=500 MB
> # Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance
Event from
> # the repository. If the length of any attribute exceeds this value, it will be truncated
when the event is retrieved.
> nifi.provenance.repository.max.attribute.length=65536
> nifi.provenance.repository.concurrent.merge.threads=2
> nifi.provenance.repository.warm.cache.frequency=1 hour
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message