nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominik Benz <dominik.b...@gmail.com>
Subject Re: session recover behaviour in nifi-jms-processor
Date Sat, 25 Feb 2017 09:27:51 GMT
Just one quick addition: I understood that Nifi prefers message duplication
over message loss (which we are fine with in principal, because we can
de-duplicate later). However, in our current situation, we have to stop the
connection after a certain time interval (mostly roughly 90minutes) due to
the following behaviour:

1) session.recover within nifi-jms-processor leads to re-delivery of
not-yet-acked messages (as described above)
2) over time, the number of re-delivered message grows, and is "piling up"
3) starting from a certain time, the "pile" of re-delivered messages has
grown from 10 or 20 to hundreds
4) in this situation, the consumers "fall behind" too much and cannot cope
with the newly arriving messages (in addition to the many not-yet-acked old
ones)
5) the JMS topic starts to have a growing number of pending messages for the
aforementioned reason
6) the JMS server side gets storage problems due to buffering the pending
messages
7) we have to disconnect to avoid JMS server crashes

This is just for clarification that the current implementation doesn't only
lead to "some" duplicates (which would be fine), but rather to more severe
problems on our side.



--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/session-recover-behaviour-in-nifi-jms-processor-tp14940p14976.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Mime
View raw message