nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: GetJMSQueue does not detect dead connections
Date Thu, 16 Jun 2016 19:15:01 GMT
Oleg - so to Chris' other comment about docs suggesting the ActiveMQ
lib path is not needed - is that a doc bug?

On Thu, Jun 16, 2016 at 3:13 PM, Oleg Zhurakousky
<ozhurakousky@hortonworks.com> wrote:
> Chris
>
> That is correct.
> The idea was to make sure that we can support multiple clients and multiple vendors since
Get/Put only supported AMQP and only one version. The new JMS support allows you to use any
JMS vendor and the only extra work we are asking you to do is to provide ConnectionFactory
JAR(s).
>
> Does that clarify?
>
> Also, yeh tests I was referring to are https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/test/java/org/apache/nifi/jms/processors
>
> Let me know if you need more help
> Cheers
> Oleg
> On Jun 16, 2016, at 3:08 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcdermott@hpe.com<mailto:chris.mcdermott@hpe.com>>
wrote:
>
> So does that mean that I cannot use the AMQ client packaged with NiFi but rather provide
my own?
>
> Sorry if I an being obtuse.
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
> https://www.storefrontremote.com
>
> On 6/16/16, 2:53 PM, "Oleg Zhurakousky" <ozhurakousky@hortonworks.com> wrote:
>
> Yes, you can probably look at the test case for it since it uses embedded AMQP.
>
> Let m know if you need more help with it.
>
> Cheers
> Oleg
> On Jun 16, 2016, at 2:50 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcdermott@hpe.com>
wrote:
>
> Thanks, Oleg.
>
> Do you have an example of how to configure the JMSConnectionFactoryProvider to work with
AMQ?
>
> The documentation says that the MQ Client Libraries path is optional with org.apache.activemq.ActiveMQConnectionFactory
but I am find that is not the case.
>
> Thanks,
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
> https://www.storefrontremote.com
>
> On 6/16/16, 1:43 PM, "Oleg Zhurakousky" <ozhurakousky@hortonworks.com> wrote:
>
> Chris
>
> Given that we are deprecating Get/PutJMS* in favor of Publish/SubscribeJMS, I’d suggest
start using those once.
>
> Cheers
> Oleg
>
>
> On Jun 16, 2016, at 1:34 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcdermott@hpe.com>
wrote:
>
> Folks,
>
> I’ve been trying to test my GetJMSQueue configuration so that it detects a dead broker
connection and fails over to an alternate broker.  When I say dead connection I mean TCP connection
that has not been closed but is no longer passing traffic.  In the real world this typically
happens when broker server crashes and so it does not reset the open connections.  For my
test case I am using iptables to block traffic.
>
> This is the connection URI I am using
>
> failover:(tcp://host2:61616,tcp://host1:61616)?randomize=false&timeout=3000&nested.soTimeout=30000&nested.soWriteTimeout=30000&startupMaxReconnectAttempts=1&maxReconnectAttempts=0
>
> They key parameters here are soTimeout=30000 and soWriteTimeout=30000
>
> These set a 30 second timeout on socket reads and writes.  I’m not sure if these are
necessary since I believe the JMSConsumer classes specifies its own timeout according to the
processor configuration.  The important thing to note is that when one of these timeouts occurs
the AMQ client does not close the connection.
>
> I believe the deficiency here is that JMSConsumer does not consider the possibility that
the connection is dead.   The problem with this is that an attempt to reconnect and failover
to an alternate broker is not made.
>
> I think the fix would involve counting the number of sequential empty responses on the
connection and then closing the connection once that number crosses some threshold.  Then
subsequent onTrigger() would cause a new connection attempt.
>
> Thoughts?
>
> Chris McDermott
>
> Remote Business Analytics
> STaTS/StoreFront Remote
> HPE Storage
> Hewlett Packard Enterprise
> Mobile: +1 978-697-5315
>
> https://www.storefrontremote.com
>
>
>
>
>

Mime
View raw message