Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
Date: Wed, 7 Jan 2015 12:09:34 +0000 (UTC)
From: "Gary Tully (JIRA)" <jira@apache.org>
To: dev@activemq.apache.org
Message-ID: <JIRA.12724267.1403920777000.27611.1420632574812@Atlassian.JIRA>
In-Reply-To: <JIRA.12724267.1403920777000@Atlassian.JIRA>
References: <JIRA.12724267.1403920777000@Atlassian.JIRA>
 <JIRA.12724267.1403920777393@arcas>
Subject: [jira] [Commented] (AMQ-5249) "cursor got duplicate" error after
 upgrade
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/AMQ-5249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14267560#comment-14267560 ] 

Gary Tully commented on AMQ-5249:
---------------------------------

@Alex - there was a change of behaviour in 5.10 - to treat duplicates from the store a as problem - in the past, they were ignored by the cursor but could get redispatched on a restart or if the audit was exhausted, they would be redispatched as duplicates. in 5.10, if the page in logic from the store gets a duplicate if does the DLQ thing. So in some ways the difference between 5.9 and 5.10 is expected.
What is unknown is what exact scenario causes the behaviour you see.
The trigger for the duplicates I tracked in the past was queue memory limit being reached with concurrent producers to the same destination, some of which were transactional. The interleaving could leave the cursor and store out of sync, such that when the cursor needed to page in from the store, it would use the wrong start point.
Your trigger seems to be related to the network bridge, which would introduce a non transacted client ack consumer and non transacted producer. Maybe you can simulate that at load without the bridge.
We need to determine the sequence of events that lead to the duplicate from the store to understand if it is legitimate or not and if not, to ensure we don't fill the DLQ in this case.

> "cursor got duplicate" error after upgrade
> ------------------------------------------
>
>                 Key: AMQ-5249
>                 URL: https://issues.apache.org/jira/browse/AMQ-5249
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.9.1, 5.10.0
>            Reporter: Rural Hunter
>
> I was using 5.9.0 and meet one problem so I tried to upgrade activemq. I tried both 5.9.1 and 5.10.0 and encouterred a same problem. I saw messages filled DLQ very quickly. I checked the clients both producer and consumer but there was no error. I checked activemq log and found the log is full of these warnings:
> 2014-06-27 23:22:09,337 | WARN  | org.apache.activemq.broker.region.cursors.QueueStorePrefetch@19117501:com.cyyun.webmon.spider.update,batchResetNeeded=false,storeHasMessages=true,size=211,cacheEnabled=true,maxBatchSize:200,hasSpace:true - cursor got duplicate: ID:211.com-52399-1400732399425-1:1:235992:1:1, 4 | org.apache.activemq.broker.region.cursors.AbstractStoreCursor | ActiveMQ Broker[localhost] Scheduler
> 2014-06-27 23:22:09,337 | WARN  | org.apache.activemq.broker.region.cursors.QueueStorePrefetch@19117501:com.xxxx.update,batchResetNeeded=false,storeHasMessages=true,size=211,cacheEnabled=true,maxBatchSize:200,hasSpace:true - cursor got duplicate: ID:nbzjjf22805-34129-1403880308671-1:1:28:1:1, 4 | org.apache.activemq.broker.region.cursors.AbstractStoreCursor | ActiveMQ Broker[localhost] Scheduler
> 2014-06-27 23:22:09,338 | WARN  | org.apache.activemq.broker.region.cursors.QueueStorePrefetch@19117501:com.xxxxx.update,batchResetNeeded=false,storeHasMessages=true,size=211,cacheEnabled=true,maxBatchSize:200,hasSpace:true - cursor got duplicate: ID:jxncxnj2-48598-1403856107346-1:1:6007:1:1, 4 | org.apache.activemq.broker.region.cursors.AbstractStoreCursor | ActiveMQ Broker[localhost] Scheduler
> 2014-06-27 23:22:09,338 | WARN  | org.apache.activemq.broker.region.cursors.QueueStorePrefetch@19117501:com.xxxx.update,batchResetNeeded=false,storeHasMessages=true,size=211,cacheEnabled=true,maxBatchSize:200,hasSpace:true - cursor got duplicate: ID:jxnc17-60227-1400730816361-1:1:149072:1:1, 4 | org.apache.activemq.broker.region.cursors.AbstractStoreCursor | ActiveMQ Broker[localhost] Scheduler
> 2014-06-27 23:22:09,339 | WARN  | org.apache.activemq.broker.region.cursors.QueueStorePrefetch@19117501:com.xxxx.update,batchResetNeeded=false,storeHasMessages=true,size=211,cacheEnabled=true,maxBatchSize:200,hasSpace:true - cursor got duplicate: ID:cyyun-46954-1403800808565-1:1:9765:1:1, 4 | org.apache.activemq.broker.region.cursors.AbstractStoreCursor | ActiveMQ Broker[localhost] Scheduler
> 2014-06-27 23:22:09,339 | WARN  | org.apache.activemq.broker.region.cursors.QueueStorePrefetch@19117501:com.xxxx.update,batchResetNeeded=false,storeHasMessages=true,size=211,cacheEnabled=true,maxBatchSize:200,hasSpace:true - cursor got duplicate: ID:ubuntu-55495-1403497638437-1:1:53086:1:1, 4 | org.apache.activemq.broker.region.cursors.AbstractStoreCursor | ActiveMQ Broker[localhost] Scheduler
> 2014-06-27 23:22:09,340 | WARN  | org.apache.activemq.broker.region.cursors.QueueStorePrefetch@19117501:com.xxxx.update,batchResetNeeded=false,storeHasMessages=true,size=211,cacheEnabled=true,maxBatchSize:200,hasSpace:true - cursor got duplicate: ID:cyyun-39030-1403880008363-1:1:70:1:1, 4 | org.apache.activemq.broker.region.cursors.AbstractStoreCursor | ActiveMQ Broker[localhost] Scheduler
> The problem mostly happens right after activemq starts and sometimes happened after activemq worked normally for a while.
> For now I have to roll back to 5.9.0 and the problem doesn't occure.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)