Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
Date: Mon, 15 Oct 2012 08:14:03 +0000 (UTC)
From: "Yuriy Sidelnikov (JIRA)" <jira@apache.org>
To: dev@activemq.apache.org
Message-ID: <2049202862.44639.1350288843669.JavaMail.jiratomcat@arcas>
In-Reply-To: <288942681.44629.1350288603081.JavaMail.jiratomcat@arcas>
Subject: [jira] [Updated] (AMQ-4107) Message order can be broken for Topic
 under a high load when topicPrefetch=1 and comsumer is slow
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


     [ https://issues.apache.org/jira/browse/AMQ-4107?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:all-tabpanel ]

Yuriy Sidelnikov updated AMQ-4107:
----------------------------------

    Description:=20
For <amq:policyEntry topic=3D">" producerFlowControl=3D"true" memoryLimit=
=3D"30mb" {color:red}topicPrefetch=3D"1"{color} blockedProducerWarningInter=
val=3D"30">

Short excerpt from TopicSubscription class:
public void add(MessageReference node) throws Exception {

=E2=80=A6..
              if ({color:red}!isFull(){color} && matched.isEmpty()  && !isS=
lave()) {
            // if maximumPendingMessages is set we will only discard messag=
es which
            // have not been dispatched (i.e. we allow the prefetch buffer =
to be filled)
       {color:red}dispatch(node);{color}                   <- Second messag=
e will go this way and might be dispatched sooner than first one.
            setSlowConsumer(false);
        } else {
=E2=80=A6=E2=80=A6.
if (matched.tryAddMessageLast(node, 10)) {    <- first message will be put =
in the VMCursor queue and might be dispatched later=20
                            break;
                        }
.....
 dispatchMatched();   <- First message won't be dispatched immediately beca=
use !isFull() is still false
}
Possible scenario as I can see it from logs:
1. First message has arrived and !isFull() is false because consumer didn't=
 take some previous message yet.
2. First message will be processed by tryAddMessageLast in VMPendingMessage=
Cursor class and will be dispatched very lately because !isFull() is still =
false.
3. Meanwhile consumer reads some previous message and !isFull() will return=
 true.   =20
4. Second message will be dispatched immediately and might be first to be d=
elivered.=20
5. Then first message is dispatched.
6. Message order is broken.


  was:
For <amq:policyEntry topic=3D">" producerFlowControl=3D"true" memoryLimit=
=3D"30mb" {color:red}topicPrefetch=3D"1"{color} blockedProducerWarningInter=
val=3D"30">

Short excerpt from TopicSubscription class:
public void add(MessageReference node) throws Exception {

=E2=80=A6..
              if ({color:red}!isFull(){color} && matched.isEmpty()  && !isS=
lave()) {
            // if maximumPendingMessages is set we will only discard messag=
es which
            // have not been dispatched (i.e. we allow the prefetch buffer =
to be filled)
            dispatch(node);                   <- Second message will go thi=
s way and might be dispatched sooner than first one.
            setSlowConsumer(false);
        } else {
=E2=80=A6=E2=80=A6.
if (matched.tryAddMessageLast(node, 10)) {    <- first message will be put =
in the VMCursor queue and might be dispatched later=20
                            break;
                        }
.....
 dispatchMatched();   <- First message won't be dispatched immediately beca=
use !isFull() is still false
}
Possible scenario as I can see it from logs:
1. First message has arrived and !isFull() is false because consumer didn't=
 take some previous message yet.
2. First message will be processed by tryAddMessageLast in VMPendingMessage=
Cursor class and will be dispatched very lately because !isFull() is still =
false.
3. Meanwhile consumer reads some previous message and !isFull() will return=
 true.   =20
4. Second message will be dispatched immediately and might be first to be d=
elivered.=20
5. Then first message is dispatched.
6. Message order is broken.


   =20
> Message order can be broken for Topic under a high load when topicPrefetc=
h=3D1 and comsumer is slow
> -------------------------------------------------------------------------=
------------------------
>
>                 Key: AMQ-4107
>                 URL: https://issues.apache.org/jira/browse/AMQ-4107
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Transport
>    Affects Versions: 5.6.0
>            Reporter: Yuriy Sidelnikov
>
> For <amq:policyEntry topic=3D">" producerFlowControl=3D"true" memoryLimit=
=3D"30mb" {color:red}topicPrefetch=3D"1"{color} blockedProducerWarningInter=
val=3D"30">
> Short excerpt from TopicSubscription class:
> public void add(MessageReference node) throws Exception {
> =E2=80=A6..
>               if ({color:red}!isFull(){color} && matched.isEmpty()  && !i=
sSlave()) {
>             // if maximumPendingMessages is set we will only discard mess=
ages which
>             // have not been dispatched (i.e. we allow the prefetch buffe=
r to be filled)
>        {color:red}dispatch(node);{color}                   <- Second mess=
age will go this way and might be dispatched sooner than first one.
>             setSlowConsumer(false);
>         } else {
> =E2=80=A6=E2=80=A6.
> if (matched.tryAddMessageLast(node, 10)) {    <- first message will be pu=
t in the VMCursor queue and might be dispatched later=20
>                             break;
>                         }
> .....
>  dispatchMatched();   <- First message won't be dispatched immediately be=
cause !isFull() is still false
> }
> Possible scenario as I can see it from logs:
> 1. First message has arrived and !isFull() is false because consumer didn=
't take some previous message yet.
> 2. First message will be processed by tryAddMessageLast in VMPendingMessa=
geCursor class and will be dispatched very lately because !isFull() is stil=
l false.
> 3. Meanwhile consumer reads some previous message and !isFull() will retu=
rn true.   =20
> 4. Second message will be dispatched immediately and might be first to be=
 delivered.=20
> 5. Then first message is dispatched.
> 6. Message order is broken.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs
For more information on JIRA, see: http://www.atlassian.com/software/jira