activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gtamir <giora.ta...@active.com>
Subject Re: 5.2.0 dequeue counter=0 pending queue stalls
Date Wed, 11 Feb 2009 22:51:22 GMT

Encountering similar situation. Found it was sufficient to just delete the
kaha store to cause listeners to start picking up messages (stop ActiveMQ
5.2 final, delete data/kr-store and restart ActiveMQ).

Switched to using Derby as a JDBC data store instead of the kaha store - no
improvement.


thomaskwan wrote:
> 
> We are using ActiveMQ 5.2.0 the latest stable release. This is what i have
> seen:
> 
> Client ID: ...
> Subscription Name: ...
> Connection ID: ...
> Pending Queue Size: 9894
> Dispatched Queue Size 0
> Dispatched Counter: 0
> Enqueue Counter: 9894
> Dequeue Counter: 0
> 
> Within the same MQ server, some consumers work, some just got stuck into
> the state that I have
> mentioned above. The active MQ log shows:
> 
> 
> 2009-01-25 14:33:14,489 [168.20.36:53300] ERROR RecoveryListenerAdapter       
> - Message id ID:hefeed1-48611-1232757655348-4:140816:-1:1:1 could not be
> recovered from the data store - already dispatched
> 2009-01-25 14:33:14,490 [168.20.36:53300] ERROR RecoveryListenerAdapter       
> - Message id ID:hefeed1-48611-1232757655348-4:140816:-1:1:1 could not be
> recovered from the data store - already dispatched
> ...
> 2009-01-25 14:33:15,428 [168.20.36:53310] ERROR DataManagerImpl               
> - Looking for key 4 but not found in fileMap: {5=data-topic-data-5 number
> = 5 , length = 2097120 refCount = 2460, 6=data-topic-data-6 number = 6 ,
> length = 2097135 refCount = 2942, 7=data-topic-data-7 number = 7 , length
> = 2097127 refCount = 2844, 8=data-topic-data-8 number = 8 , length =
> 2097111 refCount = 3308, 9=data-topic-data-9 number = 9 , length = 2097127
> refCount = 3196, 10=data-topic-data-10 number = 10 , length = 2097111
> refCount = 3341, 11=data-topic-data-11 number = 11 , length = 2097127
> refCount = 3625, 12=data-topic-data-12 number = 12 , length = 2097111
> refCount = 3602, 13=data-topic-data-13 number = 13 , length = 2097127
> refCount = 3544, 14=data-topic-data-14 number = 14 , length = 2097111
> refCount = 3412, 15=data-topic-data-15 number = 15 , length = 2097127
> refCount = 3698, 17=data-topic-data-17 number = 17 , length = 2097127
> refCount = 3612, 16=data-topic-data-16 number = 16 , length = 2097111
> refCount = 3570, 19=data-topic-data-19 number = 19 , length = 2097127
> refCount = 3588, 18=data-topic-data-18 number = 18 , length = 2097111
> refCount = 3468, 21=data-topic-data-21 number = 21 , length = 2097127
> refCount = 3322, 20=data-topic-data-20 number = 20 , length = 2097111
> refCount = 3284, 23=data-topic-data-23 number = 23 , length = 2097127
> refCount = 3336, 22=data-topic-data-22 number = 22 , length = 2097111
> refCount = 3188, 25=data-topic-data-25 number = 25 , length = 2097127
> refCount = 3270, 24=data-topic-data-24 number = 24 , length = 2097111
> refCount = 3322, 27=data-topic-data-27 number = 27 , length = 2097127
> refCount = 3240, 26=data-topic-data-26 number = 26 , length = 2097111
> refCount = 3214, 29=data-topic-data-29 number = 29 , length = 2097127
> refCount = 3290, 28=data-topic-data-28 number = 28 , length = 2097111
> refCount = 3126, 31=data-topic-data-31 number = 31 , length = 2097127
> refCount = 3382, 30=data-topic-data-30 number = 30 , length = 2097111
> refCount = 3200, 34=data-topic-data-34 number = 34 , length = 2097111
> refCount = 2928, 35=data-topic-data-35 number = 35 , length = 2097127
> refCount = 3094, 32=data-topic-data-32 number = 32 , length = 2097111
> refCount = 2866, 33=data-topic-data-33 number = 33 , length = 2097127
> refCount = 3096, 38=data-topic-data-38 number = 38 , length = 2097111
> refCount = 3202, 39=data-topic-data-39 number = 39 , length = 2097127
> refCount = 3158, 36=data-topic-data-36 number = 36 , length = 2097111
> refCount = 3146, 37=data-topic-data-37 number = 37 , length = 2097127
> refCount = 3082, 42=data-topic-data-42 number = 42 , length = 2097111
> refCount = 9917, 43=data-topic-data-43 number = 43 , length = 2097127
> refCount = 24897, 40=data-topic-data-40 number = 40 , length = 2097111
> refCount = 3144, 41=data-topic-data-41 number = 41 , length = 2097127
> refCount = 3344, 44=data-topic-data-44 number = 44 , length = 2097111
> refCount = 24650, 45=data-topic-data-45 number = 45 , length = 1051587
> refCount = 12466}
> 2009-01-25 14:33:15,429 [168.20.36:53310] ERROR MapContainerImpl              
> - Failed to get value for offset=17748, key=(4, 610628, 47), value=(4,
> 610680, 63), previousItem=102, nextItem=-1
> java.io.IOException: Could not locate data file data-topic-data-4
>         at
> org.apache.activemq.kaha.impl.data.DataManagerImpl.getDataFile(DataManagerImpl.java:129)
>         at
> org.apache.activemq.kaha.impl.data.SyncDataFileReader.readItem(SyncDataFileReader.java:65)
>         at
> org.apache.activemq.kaha.impl.data.DataManagerImpl.readItem(DataManagerImpl.java:141)
>         at
> org.apache.activemq.kaha.impl.container.MapContainerImpl.getValue(MapContainerImpl.java:481)
>         at
> org.apache.activemq.store.kahadaptor.KahaTopicReferenceStore.recoverNextMessages(KahaTopicReferenceStore.java:254)
>         at
> org.apache.activemq.store.amq.AMQTopicMessageStore.recoverNextMessages(AMQTopicMessageStore.java:59)
>         at
> org.apache.activemq.broker.region.cursors.TopicStorePrefetch.doFillBatch(TopicStorePrefetch.java:94)
> ..
> 2009-01-25 14:33:15,429 [168.20.36:53310] ERROR AbstractStoreCursor           
> - Failed to fill batch
> org.apache.activemq.kaha.RuntimeStoreException: java.io.IOException: Could
> not locate data file data-topic-data-4
>         at
> org.apache.activemq.kaha.impl.container.MapContainerImpl.getValue(MapContainerImpl.java:484)
>         at
> org.apache.activemq.store.kahadaptor.KahaTopicReferenceStore.recoverNextMessages(KahaTopicReferenceStore.java:254)
>         at
> org.apache.activemq.store.amq.AMQTopicMessageStore.recoverNextMessages(AMQTopicMessageStore.java:59)
>         at
> org.apache.activemq.broker.region.cursors.TopicStorePrefetch.doFillBatch(TopicStorePrefetch.java:94)
>         at
> org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:227)
>         at
> org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:100)
>         at
> org.apache.activemq.broker.region.cursors.StoreDurableSubscriberCursor.reset(StoreDurableSubscriberCursor.java:233)
>         at
> org.apache.activemq.broker.region.PrefetchSubscription.dispatchPending(PrefetchSubscription.java:566)
>         at
> org.apache.activemq.broker.region.PrefetchSubscription.add(PrefetchSubscription.java:150)
>         at
> org.apache.activemq.broker.region.DurableTopicSubscription.add(DurableTopicSubscription.java:197)
>         at
> org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
>         at
> org.apache.activemq.broker.region.Topic.dispatch(Topic.java:580)
>         at
> org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:427)
>         at org.apache.activemq.broker.region.Topic.send(Topic.java:365)
>         at
> org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:350)
>         at
> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:437)
>         at
> org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
>         at
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
>         at
> org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:95)
>         at
> org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:133)
>         at
> org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:450)
>         at
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:639)
>         at
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:305)
> ..
> 
> To fix this problem, we have to delete the data store and let MQ recreate
> it. My question is
> that is there a way to recover from this situation? More importantly, is
> there a way to
> prevent ourself to get into this situation.
> 
> thanks
> thomas
> 

-- 
View this message in context: http://www.nabble.com/5.2.0-dequeue-counter%3D0-pending-queue-stalls-tp21658012p21966159.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message