activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiram Chirino" <hi...@hiramchirino.com>
Subject Re: Trouble with prefetch buffer.
Date Tue, 01 Aug 2006 20:42:26 GMT
Hi Vadim,

On 7/31/06, Vadim Pesochinsky <Vadim.Pesochinskiy@mscibarra.com> wrote:
>
> Hi!
>
> I have a test case to consider. 2 servers are processing 3 submitted jobs
> with following processing times 10 min, 1 min, 1 min. This sequence should
> finish in 10 minutes as one service will pick up the 10 minutes job,
> meanwhile the other one should manage the two 1 minute jobs. Since I cannot
> set prefetchSize=0, one of the 1 minute jobs is sitting in prefetch buffer
> and the jobs are processed in 11 minutes instead of 10.

I would think that a 10 min job will sit in the prefetch buffer in 1
consumer and a 1 min job sit in the prefetch buffer of the 2nd
consumer and the last 1 min job will not be in any prefetch buffer.
It will be waiting for one of the 2 consumers prefetch buffers to
empty.

Since the prefetch buffer usage is only decreased when the message is
acknowledged, if you acknowledge the message after you have processed
it, I think it should work how you want it to work.  If you are using
AUTO-ACK, the message gets acked as soon as the message is received()
before you process it.  So then another message queues up waiting for
your 10 min processing time.  I think if you witch to using client ack
or to using transacted sessions, things should work better for you.

>
> This is simplification of the real scenario where I have about 30 consumers
> submitting jobs to 20 consumers through AMQ 4.0.1. I have following
> problems:
>
> •       Messages are sitting in prefetch buffer are not available to processors,
> which results in a lot of idle time.
> •       Order of processing is random. For some reason Job # 20 is processed after
> Job # 1500. Since senders are synchronously blocked this can result in
> time-outs.
>
> What can I do to work around these problems? Any configuration changes? How
> to set prefetchSize=0; should submit a JIRA for this?
>
> Environment & Configuration
> AMQ 4.0.1
> QUEUE_NAME?consumer.prefetchSize=1;consumer.dispatchAsync=true
>
>
> <beans xmlns="http://activemq.org/config/1.0">
>     <!-- Allows us to use system properties as variables in this
> configuration file -->
>     <bean
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>
>     <broker useJmx="true" deleteAllMessagesOnStartup="true"
> persistent="true" >
>         <!-- Use the following to configure how ActiveMQ is exposed in JMX
> -->
>
>         <managementContext>
>             <managementContext connectorPort="1099"
> jmxDomainName="org.apache.activemq"/>
>         </managementContext>
>
>         <destinationPolicy>
>             <policyMap>
>                 <policyEntries>
>                     <policyEntry topic="FIN.>">
>                         <dispatchPolicy>
>                             <strictOrderDispatchPolicy />
>                         </dispatchPolicy>
>                         <pendingMessageLimitStrategy>
>                             <constantPendingMessageLimitStrategy limit="0"/>
>                         </pendingMessageLimitStrategy>
>                         <messageEvictionStrategy>
>                             <oldestMessageEvictionStrategy />
>                         </messageEvictionStrategy>
>                     </policyEntry>
>                 </policyEntries>
>             </policyMap>
>         </destinationPolicy>
>
>         <persistenceAdapter>
>             <kahaPersistentAdaptor dir="activemq-kaha"/>
>         </persistenceAdapter>
>
>         <transportConnectors>
>             <transportConnector name="default" uri="tcp://localhost:61616"
> discoveryUri="multicast://default"/>
>         </transportConnectors>
>
>         <networkConnectors>
>             <networkConnector name="default"
> uri="static://(tcp://localhost:61616)"/>
>         </networkConnectors>
>     </broker>
> </beans>
>
> --
> View this message in context: http://www.nabble.com/Trouble-with-prefetch-buffer.-tf2029800.html#a5583273
> Sent from the ActiveMQ - User forum at Nabble.com.
>
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Mime
View raw message