nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gardella Juan Pablo (Jira)" <>
Subject [jira] [Updated] (NIFI-7563) Optimize the usage of JMS sessions and message producers
Date Wed, 01 Jul 2020 10:04:00 GMT


Gardella Juan Pablo updated NIFI-7563:

Latest PR at

> Optimize the usage of JMS sessions and message producers
> --------------------------------------------------------
>                 Key: NIFI-7563
>                 URL:
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>    Affects Versions: 1.6.0, 1.8.0, 1.7.1, 1.10.0, 1.9.2, 1.11.4
>            Reporter: Gardella Juan Pablo
>            Assignee: Gardella Juan Pablo
>            Priority: Minor
>          Time Spent: 25h 10m
>  Remaining Estimate: 47h
> Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is
required to publish 1 message to the destination {{D}} using [PublishJMS|].
The message is a flow file in the processor input queue.
> It is important to know that internally the processor is using [CachingConnectionFactory|]
to reuse objects and a [worker|]
to be able to use in thread safe manner. For JMS publishers, the default configuration is
to cache connections, sessions (only 1) and message producers.
> *Preconditions*
>  # Flowfile has either {{jms_destination}} or {{jms_replyTo}} attribute defined. Due
to NIFI-7561, it should contain the word {{queue}} or {{topic}}. Also notice {{jms_destination}}
should be ignored, as suggested at NIFI-7564. That will limit the scenario only when {{jms_replyTo}}
attribute is defined.
>  # For simplicity, the processor is the first time it processes messages.
> *Scenario*
>  # Processor picks the message. The [worker|]
is created.
>  # Connection {{C1}} and session {{S1}} are created. The [Message|]
{{M1_S1}} is created and [MessageProducer|]
{{MP_S1}} created too. Required to deliver first message at [JMSPublisher#publish|].
>  # S1 and C1 are stored in {{CachingConnectionFactory}}. The caching connection factory
is created at [|].
>  # An attempt to create a new connection and a new session are requested to the connection
factory to create destination defined in the header {{jms_destination}} at [|].
Notice the connection {{C1}} is reused although *{{S1}} is not reused* (it is required to
check internal logic in CachingConnectionFactory to understand why not). A new session {{S2}}
is created and stored in the {{CachingConnectionFactory}} as the new cached session.
>  # Message is published and {{S1}} and {{MP_S1}} are closed. As {{S1}} is not in the
cache, it is physically closed and {{MP_S1}}.
>  # At this point of time, the cached objects are {{C1}}, {{S2}}. *Ideally*, all resources
should be reused.
> The scenario if it is applied to N consecutive messages create a lot of sessions and
message producers. 
> We found this issue by adding an [Interceptor|]
to an [Apache ActiveMQ v5.x|] broker to detect
the optimal usage of resources. For example, only one message producer per connection. In
below scenario we will be created N producers for the same connection. Also in a Nifi flow
that connects a [ConsumeJMS|]
with a PublishJMS. Notice {{ConsumeJMS}} populates by default {{jms_destination}} flowfile
attribute which, if it is not removed, it is processed by {{PublishJMS}} processor (by solving
NIFI-7564 should not happen any more).

This message was sent by Atlassian Jira

View raw message