nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominik Benz <dominik.b...@gmail.com>
Subject session recover behaviour in nifi-jms-processor
Date Fri, 24 Feb 2017 14:08:14 GMT
Hi,

we're currently using Nifi to consume a relatively high-traffic JMS topic
(40-60 messages per second). 

Worked well in principle - however, we then noticed that the the outbound
rate (i.e. the number of messages we fetched) of the topic was consistently
slightly higher than its inbound rate (i.e. the actual number of messages
sent to the topic). This puzzled me, because (being the only subscriber to
the topic) I would expect inbound and outbound traffic to be identical
(given we can consume fast enough, which we can).

Digging deeper, I found that in 

  org.apache.nifi.jms.processors.JMSConsumer

the method "consume" performs a session.recover:



session.recover (as written in the comment) basically stops message delivery
and re-starts from the last non-acked message. However, I think this leads
to the following issue in high-traffic contexts:

1) several threads perform the JMS session callback in parallel
2) each callback performs a session.recover
3) during high traffic, the situation arises that the ACKs from another
thread may not (yet) have arrived at the JMS server
4) this implies that the pointer of session.recover will reconsume the
not-yet-acked message from another thread

For verification, I performed so far the following steps:

(a) manual implementation of a simplistic synchronous JMS topic consumer ->
inbound/outbound identical as expected
(b) patched nifi-jms-processors and commented out session.recover() -
inbout/outbound identical as expected

Any thoughts on this? My current impression is that session.recover in its
current usage doesn't play well together with the multi-threaded JMS
consumers. Or do I have any misconception? 

Thanks & best regards,
  Dominik



--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/session-recover-behaviour-in-nifi-jms-processor-tp14940.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Mime
View raw message