activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Takawale, Pankaj" <pankaj.takaw...@dowjones.com>
Subject Re: JDBC 300K messages stuck in queue, consumers receives messages when manually browsed the queue
Date Mon, 30 Nov 2015 17:51:38 GMT
Further update:
These are all Selector Aware Queues.
One VirtualTopic, and multiple Selector Aware queues underneath it.
I noticed one of the big queue that is not dispatching messages to the
consumers has lot of unmatched messages. I think these unmatched messages
are suppressing the dispatching of matched messages due to mixing of
priorities.



On Mon, Nov 30, 2015 at 11:36 AM, Takawale, Pankaj <
pankaj.takawale@dowjones.com> wrote:

> One more observation:
> Number of consumers being displayed on Admin UI are way more than actual
> consumers.
> Admin UI is showing incorrect? counts:
> Pending Messages: 568073
> Number of consumers 408 (I'm expecting only 69 consumers),
> Messages Enqueued 354845
> Messages Dequeued 45383
>
> Thanks
> Pankaj
>
>
> On Mon, Nov 30, 2015 at 10:02 AM, Takawale, Pankaj <
> pankaj.takawale@dowjones.com> wrote:
>
>> Yes, my messages has mixed priorities.
>>
>> I've around 100 queues, 1500 connections, 2500 consumers, and 20
>> producers. Broker processes around 6 million messages per day.
>> This problem is occurring with only couple of high-volume queues. All
>> other queues are working fine.
>>
>>
>> I drained 300K messages by writing curl script to browse the queue
>> (whenever script successfully browses queue, consumers would receive
>> messages). Now queue browsing is not working as I'm getting GATEWAY_TIMEOUT
>> messages. So around 500K messages are stuck in queue.
>>
>> Since it's a production environment, I can not test with the latest
>> version right away.
>>
>> Thanks
>> Pankaj
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Sat, Nov 28, 2015 at 12:19 AM, Tim Bain <tbain@alumni.duke.edu> wrote:
>>
>>> The bug you linked to was pretty specifically about messages with mixed
>>> priorities.  Does that describe your scenario?
>>> On Nov 27, 2015 12:58 PM, "Timothy Bish" <tabish121@gmail.com> wrote:
>>>
>>> >
>>> > Just set your store version to 6 via
>>> BrokerService#setStoreOpenWireVersion
>>> >
>>> > On 11/27/2015 02:29 PM, Takawale, Pankaj wrote:
>>> > > I tried that, but ActiveMQ 5.12 does not start with the 5.11
>>> messages in
>>> > > database.
>>> > > I got the same exception mentioned in
>>> > > https://issues.apache.org/jira/browse/AMQ-5995
>>> > >
>>> > > Thanks
>>> > > Pankaj
>>> > >
>>> > > On Fri, Nov 27, 2015 at 2:08 PM, Timothy Bish <tabish121@gmail.com>
>>> > wrote:
>>> > >
>>> > >> On 11/27/2015 01:31 PM, Takawale, Pankaj wrote:
>>> > >>> Version: 5.11.1
>>> > >>> Persistence: JDBC (Postgresql)
>>> > >>>
>>> > >>> Consumers are using Selector expressions to select messages
based
>>> on
>>> > >>> JMSXGroupID.
>>> > >>>
>>> > >>> When a queue receives burst of messages, sometimes all messages
>>> gets
>>> > >> stuck
>>> > >>> in queue, and consumers would not receive any messages.
>>> > >>>
>>> > >>> But, when I click on the queue through Admin UI, consumer apps
>>> receive
>>> > >>> bunch of messages, and queue freezes again.
>>> > >>>
>>> > >>> Is this bug related to AMQ-5853
>>> > >>> <https://issues.apache.org/jira/browse/AMQ-5853> ?
>>> > >>>
>>> > >>>
>>> > >>> Thanks,
>>> > >>> Pankaj.
>>> > >>>
>>> > >> Quick way to find out is to run with the latest release and see
what
>>> > >> happens.
>>> > >>
>>> > >> --
>>> > >> Tim Bish
>>> > >> Sr Software Engineer | RedHat Inc.
>>> > >> tim.bish@redhat.com | www.redhat.com
>>> > >> twitter: @tabish121
>>> > >> blog: http://timbish.blogspot.com/
>>> > >>
>>> > >>
>>> >
>>> >
>>> > --
>>> > Tim Bish
>>> > Sr Software Engineer | RedHat Inc.
>>> > tim.bish@redhat.com | www.redhat.com
>>> > twitter: @tabish121
>>> > blog: http://timbish.blogspot.com/
>>> >
>>> >
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message