nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <chris.mcderm...@hpe.com>
Subject Re: ConsumeJMS and Acknowledgement Mode
Date Wed, 14 Sep 2016 14:04:34 GMT
Looking at the code the ACK mode is not being set.  

   /**
     * This method essentially performs initialization of this Processor by
     * obtaining an instance of the {@link ConnectionFactory} from the
     * {@link JMSConnectionFactoryProvider} (ControllerService) and performing a
     * series of {@link ConnectionFactory} adaptations which eventually results
     * in an instance of the {@link CachingConnectionFactory} used to construct
     * {@link JmsTemplate} used by this Processor.
     */
    private void buildTargetResource(ProcessContext context) {
        if (this.targetResource == null) {
            JMSConnectionFactoryProviderDefinition cfProvider = context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class);
            ConnectionFactory connectionFactory = cfProvider.getConnectionFactory();

            UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = new UserCredentialsConnectionFactoryAdapter();
            cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory);
            cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue());
            cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue());

            this.cachingConnectionFactory = new CachingConnectionFactory(cfCredentialsAdapter);
            this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue()));

            JmsTemplate jmsTemplate = new JmsTemplate();
            jmsTemplate.setConnectionFactory(this.cachingConnectionFactory);
            jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue());
            jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue()));
 
            // set of properties that may be good candidates for exposure via configuration
            jmsTemplate.setReceiveTimeout(10000);

            this.targetResource = this.finishBuildingTargetResource(jmsTemplate);
        }

The JavaDoc for JMSTemplate says.

Default settings for JMS Sessions are "not transacted" and "auto-acknowledge". As defined
by the Java EE specification, the transaction and acknowledgement parameters are ignored when
a JMS Session is created inside an active transaction, no matter if a JTA transaction or a
Spring-managed transaction. To configure them for native JMS usage, specify appropriate values
for the "sessionTransacted" and "sessionAcknowledgeMode" bean properties.

If I understand this correctly it means that ConsumeJMS is always using auto-acknowledge which
would explain my lost files on NiFi restart.

Do you concur?

Chris McDermott
 
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
 



On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <chris.mcdermott@hpe.com>
wrote:

    Yes.
    
    Chris McDermott
     
    Remote Business Analytics
    STaTS/StoreFront Remote
    HPE Storage
    Hewlett Packard Enterprise
    Mobile: +1 978-697-5315
     
    
    
    
    On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <ozhurakousky@hortonworks.com> wrote:
    
        So you’re asking about ConsumeJMS?
        
        > On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)
<chris.mcdermott@hpe.com> wrote:
        > 
        > Hi Oleg,
        > 
        > Sorry, I wasn’t more clear.  I should have stated that ConsumeJMS does not
expose a setting for ACK mode.  Screen shot attached. So my question is which ACK mode does
it use?
        > 
        > Thanks,
        > 
        > Chris McDermott
        > 
        > Remote Business Analytics
        > STaTS/StoreFront Remote
        > HPE Storage
        > Hewlett Packard Enterprise
        > Mobile: +1 978-697-5315
        > 
        > 
        > 
        > 
        > On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <ozhurakousky@hortonworks.com>
wrote:
        > 
        >    Chris
        > 
        >    FWIW, the Get/PutJMS* are effectively deprecated processors since they only
support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS
pair.
        > 
        >    As to your question, which specific ACK mode you are inquiring about? As the
doc currently states, the auto-ACK may loose messages. Is that what you are using?
        > 
        >    Cheers
        >    Oleg
        > 
        >    On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)
<chris.mcdermott@hpe.com<mailto:chris.mcdermott@hpe.com>> wrote:
        > 
        > 
        >    Hey folks,
        > 
        >    The GetJMSQueue processor allows configuration of the Acknowledgment Mode.
        > 
        >    Acknowledgement Mode
        > 
        >    Client Acknowledge
        > 
        >    •         Client Acknowledge
        >    •         Auto Acknowledge
        > 
        >    The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to
be lost on restart of NiFi but may provide better performance than Client Acknowledge.
        > 
        > 
        >    Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS
processor?
        > 
        >    The reason I ask is that I seem to be losing messages on NiFi restart.
        > 
        >    Thanks,
        >    Chris McDermott
        > 
        >    Remote Business Analytics
        >    STaTS/StoreFront Remote
        >    HPE Storage
        >    Hewlett Packard Enterprise
        >    Mobile: +1 978-697-5315
        > 
        > 
        > 
        > 
        > 
        
        
    
    
    


Mime
View raw message