nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <ozhurakou...@hortonworks.com>
Subject Re: ConsumeJMS and Acknowledgement Mode
Date Wed, 14 Sep 2016 14:10:42 GMT
Chris

The “AUTO_ACKNOWLEDGE” (actual value 1) is the default. So although it is not being set
explicitly it is set.

We can definitely expose the other two (client ack and dups ok).
Please raise JIRA.
Cheers
Oleg

On Sep 14, 2016, at 10:04 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcdermott@hpe.com<mailto:chris.mcdermott@hpe.com>>
wrote:

Looking at the code the ACK mode is not being set.

  /**
    * This method essentially performs initialization of this Processor by
    * obtaining an instance of the {@link ConnectionFactory} from the
    * {@link JMSConnectionFactoryProvider} (ControllerService) and performing a
    * series of {@link ConnectionFactory} adaptations which eventually results
    * in an instance of the {@link CachingConnectionFactory} used to construct
    * {@link JmsTemplate} used by this Processor.
    */
   private void buildTargetResource(ProcessContext context) {
       if (this.targetResource == null) {
           JMSConnectionFactoryProviderDefinition cfProvider = context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class);
           ConnectionFactory connectionFactory = cfProvider.getConnectionFactory();

           UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = new UserCredentialsConnectionFactoryAdapter();
           cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory);
           cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue());
           cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue());

           this.cachingConnectionFactory = new CachingConnectionFactory(cfCredentialsAdapter);
           this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue()));

           JmsTemplate jmsTemplate = new JmsTemplate();
           jmsTemplate.setConnectionFactory(this.cachingConnectionFactory);
           jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue());
           jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue()));

           // set of properties that may be good candidates for exposure via configuration
           jmsTemplate.setReceiveTimeout(10000);

           this.targetResource = this.finishBuildingTargetResource(jmsTemplate);
       }

The JavaDoc for JMSTemplate says.

Default settings for JMS Sessions are "not transacted" and "auto-acknowledge". As defined
by the Java EE specification, the transaction and acknowledgement parameters are ignored when
a JMS Session is created inside an active transaction, no matter if a JTA transaction or a
Spring-managed transaction. To configure them for native JMS usage, specify appropriate values
for the "sessionTransacted" and "sessionAcknowledgeMode" bean properties.

If I understand this correctly it means that ConsumeJMS is always using auto-acknowledge which
would explain my lost files on NiFi restart.

Do you concur?

Chris McDermott

Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315




On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <chris.mcdermott@hpe.com<mailto:chris.mcdermott@hpe.com>>
wrote:

   Yes.

   Chris McDermott

   Remote Business Analytics
   STaTS/StoreFront Remote
   HPE Storage
   Hewlett Packard Enterprise
   Mobile: +1 978-697-5315




   On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <ozhurakousky@hortonworks.com<mailto:ozhurakousky@hortonworks.com>>
wrote:

       So you’re asking about ConsumeJMS?

On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcdermott@hpe.com<mailto:chris.mcdermott@hpe.com>>
wrote:

Hi Oleg,

Sorry, I wasn’t more clear.  I should have stated that ConsumeJMS does not expose a setting
for ACK mode.  Screen shot attached. So my question is which ACK mode does it use?

Thanks,

Chris McDermott

Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315




On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <ozhurakousky@hortonworks.com<mailto:ozhurakousky@hortonworks.com>>
wrote:

  Chris

  FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one
JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair.

  As to your question, which specific ACK mode you are inquiring about? As the doc currently
states, the auto-ACK may loose messages. Is that what you are using?

  Cheers
  Oleg

  On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcdermott@hpe.com<mailto:chris.mcdermott@hpe.com><mailto:chris.mcdermott@hpe.com>>
wrote:


  Hey folks,

  The GetJMSQueue processor allows configuration of the Acknowledgment Mode.

  Acknowledgement Mode

  Client Acknowledge

  •         Client Acknowledge
  •         Auto Acknowledge

  The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart
of NiFi but may provide better performance than Client Acknowledge.


  Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor?

  The reason I ask is that I seem to be losing messages on NiFi restart.

  Thanks,
  Chris McDermott

  Remote Business Analytics
  STaTS/StoreFront Remote
  HPE Storage
  Hewlett Packard Enterprise
  Mobile: +1 978-697-5315













Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message