activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Anchlia <mohitanch...@gmail.com>
Subject Re: Stuck Messages
Date Tue, 21 May 2013 16:31:37 GMT
We wrote our own java client to monitor queues. I think there is a script
provided out of the box that you might be able to use.

On Tue, May 21, 2013 at 7:36 AM, Donald Masanz <don.masanz@carters.com>wrote:

> I'm wondering if there is a way to automatically check to see if there are
> any entries in the Visibility Inbound queue.  We still have sporadic
> situations where Apache stops running.   We have found documentation on
> where this could be caused by the producerFlowControl, however we have not
> made changes to our production system.    What we would like to do is to
> write some scripts that could check that queue value so that if the queue >
> 0 then send us notification.  This past weekend MQ stopped running and we
> had over 40,000 entries that needed to process.  With that being said we
> know that we can execute AIX scripts to monitor MQ Series so I'm wondering
> if there is a way to manually monitor Apache?
>
> From: Gaurav Sharma [via ActiveMQ] [mailto:
> ml-node+s2283324n4662994h57@n4.nabble.com]
> Sent: Thursday, February 07, 2013 11:13 AM
> To: Masanz, Don
> Subject: Re: Stuck Messages
>
> Aah, I see. So, a good place to start is:
> http://activemq.apache.org/performance-tuning.html
>
> AMQ is a non-trivial distributed system and infrastructure building block.
> So, like any such system, it needs to be tuned and configured for your
> needs and it is highly configurable and flexible.
> As you are reading the perf page, pull out your activemq.xml from the
> installed edi system - mask hosts/ports/passwds and post it on here. Your
> broker seems to be running low on system resources.
>
> On Feb 7, 2013, at 9:00, Donald Masanz <[hidden
> email]</user/SendEmail.jtp?type=node&node=4662994&i=0>> wrote:
>
> > Do you have any document available on MQ.  We had this installed in
> August of 2011, they installed MQ as part of our EDI software.  I am the
> ADMIN, but no one in our staff has actually ever done any set up or
> configurations.  It was all done for us.  Therefore when you mention all
> these options, I really don't have much of a clue what you are talking
> about.
> >
> >
> > From: Gaurav Sharma [via ActiveMQ] [mailto:[hidden
> email]</user/SendEmail.jtp?type=node&node=4662994&i=1>]
>  > Sent: Thursday, February 07, 2013 10:57 AM
> > To: Masanz, Don
> > Subject: Re: Stuck Messages
> >
> > Forgot to mention - since you know what queues and topics have fast
> consumers that can keep up with producers, you can always setup nagios
> alerts (warn/critical) on 'Pending Messages'. I use my broker liveness
> monitor daemon that periodically performs synthetic transactions against
> the broker so I can auto-markdown bad apple broker nodes on the client side.
> >
> > On Feb 7, 2013, at 8:40, Gaurav Sharma <[hidden
> email]</user/SendEmail.jtp?type=node&node=4662992&i=0>> wrote:
> >
> >> Folks - when you notice the broker in a 'stuck' state, please take a
> thread and heap dump before killing the broker process and share with devs
> on here.
> >>
> >> Also:
> >> 1. What broker topology are you using?
> >> 2. How many producers/consumers?
> >> 3. Are you pooling connections, producers, sessions?
> >> 4. How large are your messages?
> >> 5. How many concurrent connections? What is the rate of message flow
> /second?
> >> 6. Are messages persistent and if so, what store?
> >> 7. What are your system/memory usage settings? How much heap and RAM on
> brokers and on clients?
> >> 8. What is the prefetch size?
> >> 9. How are you using the broker - what role is it playing in your
> system? Using queues, topics, both?
> >> 10. Have you monitored GC activity on clients and brokers?
> >> 11. What amq transport are you using and via what client?
> >>
> >> There might be more than one factors at play here. Very curious how
> restarts are 'fixing' the problem.
> >>
> >> On Feb 7, 2013, at 8:18, Donald Masanz <[hidden
> email]</user/SendEmail.jtp?type=node&node=4662992&i=1>> wrote:
> >>
> >>> I took your advice this morning when I noticed that MQ had hung up and
> was able to keep my application running while I restarted MQ.  Great tip.
>  Do you have a process that monitors MQ and notifies you if it stops
> processing, or do you have to do a manual check?
> >>>
> >>> From: tetoconsusport [via ActiveMQ] [mailto:[hidden
> email]</user/SendEmail.jtp?type=node&node=4662992&i=2>]
> >>> Sent: Wednesday, February 06, 2013 4:52 PM
> >>> To: Masanz, Don
> >>> Subject: Re: Stuck Messages
> >>>
> >>> Did you find a solution for this? We are running activemq 5.7 and have
> had this issue as well. If we restart activemq everything works however, we
> have not been able to find the root cause.
> >>> ________________________________
> >>> If you reply to this email, your message will be added to the
> discussion below:
> >>>
> >>> NAML<
> http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context:
> http://activemq.2283324.n4.nabble.com/Stuck-Messages-tp4662092p4662985.html
> >>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >
> > ________________________________
> > If you reply to this email, your message will be added to the discussion
> below:
> >
> > NAML<
> http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >
> >
> >
> >
> > --
> > View this message in context:
> http://activemq.2283324.n4.nabble.com/Stuck-Messages-tp4662092p4662993.html
> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://activemq.2283324.n4.nabble.com/Stuck-Messages-tp4662092p4662994.html
> To unsubscribe from Stuck Messages, click here<
> http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4662092&code=ZG9uLm1hc2FuekBjYXJ0ZXJzLmNvbXw0NjYyMDkyfC0yMzQzNTczNA==
> >.
> NAML<
> http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Stuck-Messages-tp4662092p4667280.html
>  Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message