nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bmichaud <ben_mich...@optum.com>
Subject Partition closed in MergeContent bins
Date Tue, 08 Nov 2016 19:09:58 GMT
I am using NiFi 1.0.0. I have a MergeContent merging to 6 bins using a
correlation attribute that will have at most 6 values. 
<http://apache-nifi-developer-list.39713.n7.nabble.com/file/n13851/MergeContent.png>


It is directly preceded by a custom processor that is set for 12 concurrent
tasks. I am getting this error repeatedly in some of my installations,
especially when under high load (about 2 million messages every 5 minutes).
Restarting only solves the problem temporarily. 

Here is the error: 

2016-11-08 13:15:08,451 WARN [Timer-Driven Process Thread-3]
o.a.n.processors.standard.MergeContent
MergeContent[id=12054904-d034-49fa-e6b6-b4e99d0d488a] Processor
Administratively Yielded for 1 sec due to processing failure
2016-11-08 13:15:08,451 WARN [Timer-Driven Process Thread-3]
o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding
MergeContent[id=12054904-d034-49fa-e6b6-b4e99d0d488a] due to uncaught
Exception: java.lang.IllegalStateException: Partition is closed
2016-11-08 13:15:08,452 WARN [Timer-Driven Process Thread-3]
o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.IllegalStateException: Partition is closed
        at
org.wali.MinimalLockingWriteAheadLog$Partition.update(MinimalLockingWriteAheadLog.java:906)
~[nifi-write-ahead-log-1.0.0.jar:1.0.0]
        at
org.wali.MinimalLockingWriteAheadLog.update(MinimalLockingWriteAheadLog.java:230)
~[nifi-write-ahead-log-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.swapFlowFilesOut(WriteAheadFlowFileRepository.java:323)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.FileSystemSwapManager.swapOut(FileSystemSwapManager.java:129)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.StandardFlowFileQueue.writeSwapFilesIfNecessary(StandardFlowFileQueue.java:579)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.StandardFlowFileQueue.put(StandardFlowFileQueue.java:282)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.repository.StandardProcessSession.rollback(StandardProcessSession.java:949)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.repository.StandardProcessSession.rollback(StandardProcessSession.java:876)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.repository.StandardProcessSession.rollback(StandardProcessSession.java:871)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at
org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:247)
~[nifi-processor-utils-1.0.0.jar:1.0.0]
        at
org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:190)
~[nifi-processor-utils-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064)
~[nifi-framework-core-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.0.0.jar:1.0.0]
        at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
[nifi-framework-core-1.0.0.jar:1.0.0]
        at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_65]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_65]
        at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_65]
        at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_65]
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_65]
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_65]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]




--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Partition-closed-in-MergeContent-bins-tp13851.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Mime
View raw message