activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiram Chirino" <hi...@hiramchirino.com>
Subject Re: problems with ActiveMQ with large number of producers/consumers
Date Mon, 03 Jul 2006 16:54:28 GMT
I Igor,

I just recently fixed a issue where a consumer could be created and not
properly cleaned up if an async exception occured.  See
http://issues.apache.org/activemq/browse/AMQ-724.  I have a feeling that
this could have been the cause of the hung consumers.

Could you retest against a subsequent 4.1-SNAPSHOT or 4.0.2-SNAPSHOT?

Thanks!

On 7/3/06, Igor Bogicevic <caboom@gmail.com> wrote:
>
> Hi James,
>
> well, it doesn't depend on the prefetch value (I have tried with 1000
> and 2000 prefetch messages, and without prefetching at all), and what
> happens is this:
> 1) create about 60 producers and 60 consumers (I have attached the
> small and ugly test I used - basically I've forked about 60 processes
> on each side, which means they had separate sessions and connections)
> 2) if you cut out the shutdown hook in this test, which handles
> cleanup of connections and sessions, and you kill large amount of
> producers/consumers this queue will be blocked in the terms that
> neither producers or consumers will be able to either send or receive
> messages on this queue - sometimes this happens for a period of a
> 20-30s and very often for good (>10h, that's the longest period i've
> left it before i've restarted broker).
> 3) in case there is shutdown hook with properly closing sessions and
> connections, this is very hard to repeat, and in case of 4.0.0 i've
> managed to block the queue with repetitve starting/killing large
> number of producers and consumers, but i've never managed to reproduce
> this with 4.0.1
> 4) i could see a large number of hanging connections (jmx console
> showed much more connections than there actually were)
> 4) after restarting broker all works like a charm
>
> Thanks,
> Igor
>
> On 7/3/06, James Strachan <james.strachan@gmail.com> wrote:
> > By 'blocking the queue' do you just mean messages were not dispatched
> > to consumers? If so how many messages were on the queue, how many
> > connections & consumers did the broker think were open & what were the
> > prefetch values? If you are having trouble of dead connections not
> > being closed down agressively enough then your issue could just be an
> > effect of messages being placed into prefetch buffers for clients that
> > have not been detected as failed yet? Do you have any kind of JUnit
> > test case that can replicate your issue?
> >
> > On 7/3/06, Igor Bogicevic <caboom@gmail.com> wrote:
> > > I am using activemq 4.0.1 (I've did tests on the both 4.0.0 and 4.0.1)
> > > and I have attached activemq.xml (basically, almost out of the box
> > > configuration). I am using persistent queues, and I have tried with
> > > prefetching and transactions, but that didn't really change the
> > > behavior.
> > >
> > > Thanks,
> > > Igor
> > >
> > > On 7/3/06, James Strachan <james.strachan@gmail.com> wrote:
> > > > On 6/30/06, Igor Bogicevic <caboom@gmail.com> wrote:
> > > > > I also forgot to answer on this thread... this was not the problem
> > > > > related to the lack of the memory, it's related to hanging
> > > > > connections, i've set up the broker with the large amount of
> memory
> > > > > (>2GB) in both broker configuration and via jconsole and problem
> was
> > > > > defintely in the connections which were not cleaned up - for
> unknown
> > > > > reason to me. it was easy to block queue with large number of
> > > > > un-properly closed sessions and relatively hard with properly
> closed
> > > > > connections in a shutdown hook.
> > > >
> > > > Using what version and what quality of service? e.g. are we talking
> > > > 4.0.1 and non-persistent queues? What was your XML config file?
> > > >
> > > > --
> > > >
> > > > James
> > > > -------
> > > > http://radio.weblogs.com/0112098/
> > > >
> > >
> > >
> > >
> >
> >
> > --
> >
> > James
> > -------
> > http://radio.weblogs.com/0112098/
> >
>
>
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message