activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Grobmeier <grobme...@gmail.com>
Subject Re: ActiveMQ freeze
Date Fri, 21 May 2010 08:30:36 GMT
Hello all,

we have meanwhile disconnected the master/slave configuration and are
running on only one instance. However, we have the same problem. Two
times in the middle of the night all consumers are gone and then
message processing stops. Before we had the consumers still there but
doing nothing. Without the slave consumers are leaving and the webapp
shows zero of them.

Attached is a thread dump. Any ideas why this happens? I am not having
a clue anymore :-(

Christian

On Wed, May 19, 2010 at 3:34 PM, Christian Grobmeier
<grobmeier@gmail.com> wrote:
> We are not aware of any consumers having trouble. No exception or
> similar. We use one connection and several consumers on the queues.
>
> But we have enabled the master/slave only a short while before these
> problems came up. Before this configuration, we had zero problems at
> all (only with KahaDB but this issue seems to be history).
>
> My feeling is now that there is something going wrong with this
> configuration. Maybe the slave isn't writing and then the master is
> blocked. I don't know why the slave isn't writing. It looked well.
>
> For the time being we stopped master/slave config and only work with a
> single instance. My expectation is that the problems are gone then.
> However, a failover would be highly appreciated
>
>
>
> On Wed, May 19, 2010 at 3:19 PM, Gary Tully <gary.tully@gmail.com> wrote:
>> The master is blocked on a write, is some consumer having trouble? Would be
>> interested to know the state of the process on the other end of that
>> connection.
>> That pending write is blocking the next message dispatch on that connection.
>>
>>
>> "ActiveMQ Connection Dispatcher: /127.0.0.1:59497" daemon prio=10
>> tid=0x00000000486fa000 nid=0xcd0 runnable
>> [0x000000004152c000..0x000000004152ca90]
>>   java.lang.Thread.State: RUNNABLE
>> at java.net.SocketOutputStream.socketWrite0(Native Method)
>> at java.net.SocketOutputStream.socketWrite(Unknown Source)
>> at java.net.SocketOutputStream.write(Unknown Source)
>> at
>> org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
>> at java.io.DataOutputStream.flush(Unknown Source)
>> at
>> org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:181)
>> at
>> org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:243)
>> - locked <0x00002aaadca9dc98> (a java.util.concurrent.atomic.AtomicBoolean)
>> at
>> org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:85)
>> at
>> org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:104)
>> at
>> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
>> - locked <0x00002aaadca9ded8> (a java.lang.Object)
>> at
>> org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1224)
>> at
>> org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:802)
>> at
>> org.apache.activemq.broker.TransportConnection.iterate(TransportConnection.java:838)
>> at
>> org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
>> at
>> org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:36)
>>
>> On 19 May 2010 14:02, Christian grobmeier <grobmeier@gmail.com> wrote:
>>
>>>
>>> I am a collegue of Andreas. It happened again on tuesday but again we don't
>>> know the reasons. Attached are two thread dumps. I cannot see anything in
>>> it, but maybe you are seeing more.
>>> http://old.nabble.com/file/p28607920/dumps.zip dumps.zip
>>>
>>> We had also problems with KahaDB before which forced us to upgrade to this
>>> snapshot. But now I am tending to go back to Version 5.3. Are there other
>>> reports of this kind with this 5.4.x series?
>>>
>>> Christian
>>>
>>>
>>> Gary Tully wrote:
>>> >
>>> > If you can capture some thread dumps of both the master and slave
>>> brokers,
>>> > ideally a small series, with  few seconds between each this would help
>>> > determine what is going on.
>>> >
>>> > Obviously, if this is repeatable, please open a jira issue to track it
>>> and
>>> > add your test case if you can.
>>> >
>>> >
>>> > On 17 May 2010 11:51, Andreas Prüller <prueller@aracom.de> wrote:
>>> >
>>> >> Hello,
>>> >>
>>> >>
>>> >>
>>> >> currently I have an ActiveMQ 5.4-SNAPSHOT running for a test. It is
as
>>> >> pure
>>> >> master-slave configured.
>>> >>
>>> >> Last Friday the whole system stopped working – I mean no new messages
>>> >> were
>>> >> enqueued, no messages were dequeued. No exception was thrown, no
>>> warning,
>>> >> nothing. It just stopped working (and there were some hundred messages
>>> on
>>> >> the queues which had to be processed).
>>> >>
>>> >> In the log file of the ActiveMQ (master and slave) I can see it was
>>> still
>>> >> running, as it cleaned up the kahadb regularly.
>>> >>
>>> >> Only solution was at the end to stop the whole system and restart it.
>>> >>
>>> >>
>>> >>
>>> >> Anybody any idea what happened there?
>>> >>
>>> >>
>>> >>
>>> >> Hope somebody can help.
>>> >>
>>> >> Best,
>>> >>
>>> >> Andreas
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > http://blog.garytully.com
>>> >
>>> > Open Source Integration
>>> > http://fusesource.com
>>> >
>>> >
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/ActiveMQ-freeze-tp28581838p28607920.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>> --
>> http://blog.garytully.com
>>
>> Open Source Integration
>> http://fusesource.com
>>
>

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message