activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Davies <rajdav...@gmail.com>
Subject Re: ConcurrentModificationException while closing consumer
Date Wed, 11 Feb 2009 07:02:10 GMT
just for clarification - do you mean all messages delivered to the the  
MessageConsumer  would be acked  on WINDOW_ACK instead of Session wide ?

On 10 Feb 2009, at 23:42, bonnyr wrote:

>
> Rob,
>
> Can I hijack this issue and bring back an issue I have filed in Jira  
> and is
> assigned to you (AMQ-1916) and
> that is the issue of a new ack mode (I called it WINDOW_ACK) that is  
> a mix
> between CLIENT_ACK and
> INDIV_ACK in that the acknowledgement is related to the message but  
> unlike
> in CLIENT_ACK, does not
> ack all messages in a session, but rather all messages delivered  
> before the
> acked message (requires
> ordering of messages and tracking them by the broker but I think the
> building blocks are there)
>
> any idea if/when this will be implemented?
>
> Cheers,
>
> Bonny
>
> rajdavies wrote:
>>
>> Bonny - agreed - looks like that bit of code has never been
>> synchronized - will add a fix ...
>>
>> cheers,
>>
>> Rob
>>
>> Rob Davies
>> http://fusesource.com
>> http://rajdavies.blogspot.com/
>>
>>
>> On 9 Feb 2009, at 23:14, bonnyr wrote:
>>
>>>
>>> Gary,
>>>
>>> Why would the collection be used in CLIENT_ACK mode then?
>>>
>>> In our application we've got to work in this mode since there's a
>>> processing
>>> chain
>>> that may take a while between delivery of the message and its ack,  
>>> so
>>> chaging this
>>> mode is not possible for our use case.
>>>
>>> We're aware of the fact that CLIENT_ACK actually acks all the
>>> messages in a
>>> session and
>>> it's not really related to the message itself, but that's a story  
>>> for
>>> another day and another
>>> version of AMQ...
>>>
>>> However, wouldn't static analysis of the code be sufficient in this
>>> case?
>>> The code segment
>>> above is the only thing that is not protected by synchronization,
>>> whilst
>>> every other use of
>>> this collection is.
>>>
>>> Cheers,
>>>
>>> Bonny
>>>
>>>
>>> Gary Tully wrote:
>>>>
>>>>> I'm not quite able to cause this to happen in a simple test case.
>>>>> Perhaps
>>>>> this is to do with some
>>>>> thread timing/scheduling issues in the simple test case vs our
>>>>> application.
>>>>>
>>>> Yea, that is fairly typical.
>>>>
>>>>> However, are you in a
>>>>> position to explain what the deliveredMessage collection is used
>>>>> for? Is
>>>>> it
>>>>> not used in all connection/
>>>>> ack modes?
>>>>>
>>>> Sure, the deliveredMessage list is used to generate acks in the  
>>>> Auto
>>>> ack mode. In addition,
>>>> with the consumer optimizeAcknowledge option, there is more  
>>>> likely to
>>>> be some outstanding
>>>> delivered messages that require acks. On close, any outstanding  
>>>> acks
>>>> will require delivery so
>>>> you may run into contention with concurrent delivery and close
>>>> execution as you suggest.
>>>>
>>>> May I suggest using AUTO_ACK and additionally using
>>>> optimizeAcknowledge if needed.
>>>>
>>>>> Perhaps if I knew what the behaviour should be like, I'd be able
>>>>> to construct a test
>>>>> case that simulate our environment.
>>>>>
>>>> Hopefully :-)
>>>>
>>>> Gary.
>>>>
>>>>
>>>>> Cheers,
>>>>>
>>>>> Bonny
>>>>>
>>>>>
>>>>> Gary Tully wrote:
>>>>>>
>>>>>> Hi Bonny,
>>>>>> that looks like a bug indeed, should be easy to replicate in a
>>>>>> Junit
>>>>>> tests case I think. Could you raise a jira issue for this and if
>>>>>> you
>>>>>> have some tests code that demonstrates, please include it.
>>>>>>
>>>>>> for more info see: http://activemq.apache.org/contributing.html
>>>>>>
>>>>>> Thanks,
>>>>>> Gary.
>>>>>>
>>>>>> 2009/2/6 bonnyr <bonnyr@optusnet.com.au>:
>>>>>>>
>>>>>>> AMQ 5.1 (but problem exists in the sources of AMQ-5.2 as of 

>>>>>>> today)
>>>>>>>
>>>>>>> My setup:
>>>>>>> * Broker is configured with a single queue, full with messages,
>>>>>>> on a
>>>>>>> host
>>>>>>> accessible via the network.
>>>>>>> * Application configured with a single consumer, connected to
a
>>>>>>> single
>>>>>>> sesssion, running in its own thread.
>>>>>>> * ActiveMQ delivers lots of messages using one of the
>>>>>>> ActiveMQSessionTask
>>>>>>> threads.
>>>>>>> * Session configured as CLIENT_ACKNOLEDGE
>>>>>>>
>>>>>>>
>>>>>>> Since the queue is full of messages, delivery of these messages
>>>>>>> happen
>>>>>>> as
>>>>>>> fast as the network
>>>>>>> can deliver and the AMQ thread is invoking the onMessage with
>>>>>>> each new
>>>>>>> message. These messages
>>>>>>> are then processed by the consumer thread. The consumer thread
>>>>>>> then
>>>>>>> decides
>>>>>>> to close the connection
>>>>>>> and the following ensues:
>>>>>>> [code]
>>>>>>> java.util.ConcurrentModificationException
>>>>>>>      at
>>>>>>> java.util.LinkedList
>>>>>>> $ListItr.checkForComodification(LinkedList.java:617)
>>>>>>>      at java.util.LinkedList$ListItr.next(LinkedList.java:552)
>>>>>>>      at
>>>>>>> org
>>>>>>> .apache
>>>>>>> .activemq
>>>>>>> .ActiveMQMessageConsumer.dispose(ActiveMQMessageConsumer.java:

>>>>>>> 663)
>>>>>>>      at
>>>>>>> org
>>>>>>> .apache
>>>>>>> .activemq
>>>>>>> .ActiveMQMessageConsumer.close(ActiveMQMessageConsumer.java:583)
>>>>>>>      at
>>>>>>> com.xxx.app..AMsgQueueConsumer.doClose(AMsgQueueConsumer.java:

>>>>>>> 351)
>>>>>>>      at com.xxx.app.AMsgQueueConsumer.suspend
>>>>>>>
>>>>>>>      ... snip ...
>>>>>>>
>>>>>>> [/code]
>>>>>>>
>>>>>>> This happens because AMQ is busy delivering messages and there
>>>>>>> is a
>>>>>>> collection [b]deliveredMessages[/b] that is
>>>>>>> not protected by synchronisation in exactly one place
>>>>>>> (everywhere else
>>>>>>> it
>>>>>>> is...) which is executing the
>>>>>>> following code (in the dispose method):
>>>>>>> [code]
>>>>>>> ... snip ...
>>>>>>>          if (session.isClientAcknowledge()) {
>>>>>>>              if (!this.info.isBrowser()) {
>>>>>>>                  // rollback duplicates that aren't acknowledged
>>>>>>>                  for (MessageDispatch old : deliveredMessages)
{
>>>>>>>                      session.connection.rollbackDuplicate(this,
>>>>>>> old.getMessage());
>>>>>>>                  }
>>>>>>>              }
>>>>>>>          }
>>>>>>> ... snip ...
>>>>>>> [/code]
>>>>>>>
>>>>>>> Since the code iterates over the collection, and the iterator
>>>>>>> checks
>>>>>>> for
>>>>>>> modifications to the underlying collection, and since the AMQ
>>>>>>> message
>>>>>>> delivery thread has managed to sneak in a couple more messages
>>>>>>> while the consumer thread attempted to close the connection,
the
>>>>>>> problem
>>>>>>> occurs (it could be that
>>>>>>> the for syntax hides the explicit iterator calls, but...)
>>>>>>>
>>>>>>>
>>>>>>> Is this an ommission or is there a reason for not synchronising
>>>>>>> access
>>>>>>> to
>>>>>>> this collection? If it's not a bug
>>>>>>> then how should the consumer be disconnected?
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Bonny
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>> http://www.nabble.com/ConcurrentModificationException-while-closing-consumer-tp21867250p21867250.html
>>>>>>> Sent from the ActiveMQ - User mailing list archive at  
>>>>>>> Nabble.com.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> http://blog.garytully.com
>>>>>>
>>>>>> Open Source SOA
>>>>>> http://FUSESource.com
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/ConcurrentModificationException-while-closing-consumer-tp21867250p21907669.html
>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> -- 
>>>> http://blog.garytully.com
>>>>
>>>> Open Source SOA
>>>> http://FUSESource.com
>>>>
>>>>
>>>
>>> -- 
>>> View this message in context:
>>> http://www.nabble.com/ConcurrentModificationException-while-closing-consumer-tp21867250p21924323.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
> -- 
> View this message in context: http://www.nabble.com/ConcurrentModificationException-while-closing-consumer-tp21867250p21945368.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>


Mime
View raw message