activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Furness (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
Date Mon, 23 Jan 2012 22:23:49 GMT

    [ https://issues.apache.org/jira/browse/AMQ-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191550#comment-13191550
] 

James Furness commented on AMQ-3676:
------------------------------------

Ahh great, so there was something missing from my configuration - thanks for clarifying that.
Wasn't aware that the ConnectionFactory prefetch size on the consumer also needs to be set.

Would it be useful/possible to detect this situation (PolicyEntry prefetch < ConnectionFactory
prefetch) and alert? In the normal situation of individual acks I guess the mismatch wouldn't
have a significant effect, but for OptimiseAcknowledge it obviously causes the consumers to
grind to a halt.
                
> Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent
messages from being acknowledged
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3676
>                 URL: https://issues.apache.org/jira/browse/AMQ-3676
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.5.1
>         Environment: ActiveMQ: 5.5.1-fuse-01-13
> JVM: 1.6.0_26-b03-383-11A511
>            Reporter: James Furness
>         Attachments: ActiveMQOptimiseAcknowledgeManualTest.java
>
>
> Hi,
> Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've
missed in terms of the configuration but would be good to get it clarified.
> Our use case is a topic with very fast flowing data, we want all consumers to get messages
with a minimum of latency and would prefer that messages get conflated rather than building
up a large queue and hence delays.
> We therefore have the following configuration:
>  - Low prefetch size (100)
>  - Low ConstantPendingMessageLimitStrategy (100)
>  - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)
> Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable
OptimizeAcknowledge in the hopes that this would further improve latency.
> Following that was seeing that consumers receive a burst of messages, then become a slow
consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy
in place which was initially masking the problem as the reconnect caused messages to flow
again. However removing this causes the consumers to receive a burst of messages then the
flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize
in JMX), so looks like nothing is getting acked. 
> From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch
is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available),
a batch of acks should be sent.
> Have created a test case to reproduce this (attached):
> {code:title=testWithoutOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> FastConsumer: Receive 50
> Publisher: Send 100
> FastConsumer: Receive 100
> Publisher: Send 150
> FastConsumer: Receive 150
> testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58,
59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81,
82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103,
104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121,
122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,
140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157,
158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175,
176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193,
194, 195, 196, 197, 198, 199]
> testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34,
35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57,
58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80,
81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102,
103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120,
121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138,
139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156,
157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174,
175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192,
193, 194, 195, 196, 197, 198, 199]
> {code}
> {code:title=testWithOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> Publisher: Send 100
> Publisher: Send 150
> testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,
14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,
60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82,
83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122,
123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158,
159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176,
177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194,
195, 196, 197, 198, 199]
> testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> java.lang.AssertionError: 
> Expected :200
> Actual   :50
> {code}
> Cheers,
> James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message