nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <chris.mcderm...@hpe.com>
Subject Re: GetJMSQueue does not detect dead connections
Date Thu, 16 Jun 2016 19:55:20 GMT
I’ll do it.  Thanks.

Chris McDermott
 
Remote Business Analytics
STaTS/StoreFront Remote
HPE Storage
Hewlett Packard Enterprise
Mobile: +1 978-697-5315
 
https://www.storefrontremote.com

On 6/16/16, 3:19 PM, "Oleg Zhurakousky" <ozhurakousky@hortonworks.com> wrote:

>Yes, that is documentation bug.
>
>Chris would you mind raising a JIRA or I can do it.
>
>Cheers
>Oleg
>
>> On Jun 16, 2016, at 3:15 PM, Joe Witt <joe.witt@gmail.com> wrote:
>> 
>> Oleg - so to Chris' other comment about docs suggesting the ActiveMQ
>> lib path is not needed - is that a doc bug?
>> 
>> On Thu, Jun 16, 2016 at 3:13 PM, Oleg Zhurakousky
>> <ozhurakousky@hortonworks.com> wrote:
>>> Chris
>>> 
>>> That is correct.
>>> The idea was to make sure that we can support multiple clients and multiple vendors
since Get/Put only supported AMQP and only one version. The new JMS support allows you to
use any JMS vendor and the only extra work we are asking you to do is to provide ConnectionFactory
JAR(s).
>>> 
>>> Does that clarify?
>>> 
>>> Also, yeh tests I was referring to are https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/test/java/org/apache/nifi/jms/processors
>>> 
>>> Let me know if you need more help
>>> Cheers
>>> Oleg
>>> On Jun 16, 2016, at 3:08 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)
<chris.mcdermott@hpe.com<mailto:chris.mcdermott@hpe.com>> wrote:
>>> 
>>> So does that mean that I cannot use the AMQ client packaged with NiFi but rather
provide my own?
>>> 
>>> Sorry if I an being obtuse.
>>> 
>>> Chris McDermott
>>> 
>>> Remote Business Analytics
>>> STaTS/StoreFront Remote
>>> HPE Storage
>>> Hewlett Packard Enterprise
>>> Mobile: +1 978-697-5315
>>> 
>>> https://www.storefrontremote.com
>>> 
>>> On 6/16/16, 2:53 PM, "Oleg Zhurakousky" <ozhurakousky@hortonworks.com>
wrote:
>>> 
>>> Yes, you can probably look at the test case for it since it uses embedded AMQP.
>>> 
>>> Let m know if you need more help with it.
>>> 
>>> Cheers
>>> Oleg
>>> On Jun 16, 2016, at 2:50 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)
<chris.mcdermott@hpe.com> wrote:
>>> 
>>> Thanks, Oleg.
>>> 
>>> Do you have an example of how to configure the JMSConnectionFactoryProvider to
work with AMQ?
>>> 
>>> The documentation says that the MQ Client Libraries path is optional with org.apache.activemq.ActiveMQConnectionFactory
but I am find that is not the case.
>>> 
>>> Thanks,
>>> 
>>> Chris McDermott
>>> 
>>> Remote Business Analytics
>>> STaTS/StoreFront Remote
>>> HPE Storage
>>> Hewlett Packard Enterprise
>>> Mobile: +1 978-697-5315
>>> 
>>> https://www.storefrontremote.com
>>> 
>>> On 6/16/16, 1:43 PM, "Oleg Zhurakousky" <ozhurakousky@hortonworks.com>
wrote:
>>> 
>>> Chris
>>> 
>>> Given that we are deprecating Get/PutJMS* in favor of Publish/SubscribeJMS, I’d
suggest start using those once.
>>> 
>>> Cheers
>>> Oleg
>>> 
>>> 
>>> On Jun 16, 2016, at 1:34 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)
<chris.mcdermott@hpe.com> wrote:
>>> 
>>> Folks,
>>> 
>>> I’ve been trying to test my GetJMSQueue configuration so that it detects a
dead broker connection and fails over to an alternate broker.  When I say dead connection
I mean TCP connection that has not been closed but is no longer passing traffic.  In the real
world this typically happens when broker server crashes and so it does not reset the open
connections.  For my test case I am using iptables to block traffic.
>>> 
>>> This is the connection URI I am using
>>> 
>>> failover:(tcp://host2:61616,tcp://host1:61616)?randomize=false&timeout=3000&nested.soTimeout=30000&nested.soWriteTimeout=30000&startupMaxReconnectAttempts=1&maxReconnectAttempts=0
>>> 
>>> They key parameters here are soTimeout=30000 and soWriteTimeout=30000
>>> 
>>> These set a 30 second timeout on socket reads and writes.  I’m not sure if
these are necessary since I believe the JMSConsumer classes specifies its own timeout according
to the processor configuration.  The important thing to note is that when one of these timeouts
occurs the AMQ client does not close the connection.
>>> 
>>> I believe the deficiency here is that JMSConsumer does not consider the possibility
that the connection is dead.   The problem with this is that an attempt to reconnect and failover
to an alternate broker is not made.
>>> 
>>> I think the fix would involve counting the number of sequential empty responses
on the connection and then closing the connection once that number crosses some threshold.
 Then subsequent onTrigger() would cause a new connection attempt.
>>> 
>>> Thoughts?
>>> 
>>> Chris McDermott
>>> 
>>> Remote Business Analytics
>>> STaTS/StoreFront Remote
>>> HPE Storage
>>> Hewlett Packard Enterprise
>>> Mobile: +1 978-697-5315
>>> 
>>> https://www.storefrontremote.com
>>> 
>>> 
>>> 
>>> 
>>> 
>> 
>

Mime
View raw message