activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yannick Malins (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQ-5274) Stuck messages and CPU churn when aborted transacted message expires
Date Fri, 11 Jul 2014 14:33:05 GMT

    [ https://issues.apache.org/jira/browse/AMQ-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14058838#comment-14058838
] 

Yannick Malins commented on AMQ-5274:
-------------------------------------

see attached log - this seems to be due to the "sync with message expiry processing" if clause
in queuesubscription.acknowledge()

it seems that if the message is expired it is not removed - but that the expiry doesn't remove
it either

> Stuck messages and CPU churn when aborted transacted message expires
> --------------------------------------------------------------------
>
>                 Key: AMQ-5274
>                 URL: https://issues.apache.org/jira/browse/AMQ-5274
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.8.0, 5.9.0, 5.9.1, 5.10.0
>         Environment: win64, RHEL11
>            Reporter: Yannick Malins
>            Priority: Critical
>         Attachments: AMQ-5274.zip, logs_extract.txt
>
>
> The test case is a simple producer/consumer:
> Producer: 20 messages are injected, with a timeout of 10s.
> Consumer: The redelivery policy is set to 0 retries (the issue exists with other values).
The consumer uses transactions and throws a runtime exception on each message received.
> queue stats show 20 enqueue, 19 dequeue, 1 pending
> DLQ stat show 20 enqueue: all 20 messages go to DLQ, IDs ending in 1-10 for failing,
11-20 for expiry (approx)
> the pending item (ID ending in 10) is a "ghost message" , and remains stuck indefinitely
in queue.test
> if you browse, the message is not shown
> A) if you restart the broker, after a short while the message is cleaned:
> jvm 1    |  WARN | Duplicate message add attempt rejected. Destination: QUEUE://ActiveMQ.DLQ,
Message id: ID:REDACTED-52872-1405079629779-1:1:1:1:10
> jvm 1    |  WARN | org.apache.activemq.broker.region.cursors.QueueStorePrefetch@5b427f3c:ActiveMQ.DLQ,batchResetNeeded=false,storeHasMessages=true,size=0,cacheEnabled=true,maxBatchSize:20,hasSpace:tru
> e - cursor got duplicate: ID:REDACTED--52872-1405079629779-1:1:1:1:10, 4
> jvm 1    |  WARN | duplicate message from store ID:REDACTED--52872-1405079629779-1:1:1:1:10,
redirecting for dlq processing
> B) if you purge, ActiveMQ logs a warning: "WARN | queue://queue.test after purge complete,
message count stats report: 1"
> 	the queue is marked as being empty.
> 	however if you restart the broker, the message re-appears shorty, before being cleaned
as above
> 	
> 	
> SUPPLEMENTARY: with activeMQ 5.9.0 and above , if you run the injection several times,
the CPU usage of ActiveMQ climbs drastically until the queue is purged.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message