activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: 5.3 question and server upgrade question...
Date Wed, 13 Nov 2013 23:27:53 GMT
can you try a different ack mode, like clientack or using transactions
- the prefetch will be deferred till the ack which will be later than
in the auto ack case. Also, in the transacted case, use the
destination policy prefetchExtension=false

On 13 November 2013 14:54, Ned Wolpert <ned.wolpert@imemories.com> wrote:
> Did anyone have an idea into what I could do different to route messages to
> idle consumers?  Just came into the same situation this morning where a
> queue has 1 message processing on one consumer, one message waiting, and 15
> idle consumers.  (See notes below for my current configs)
>
>
> On Wed, Nov 6, 2013 at 9:40 AM, Ned Wolpert <ned.wolpert@imemories.com>wrote:
>
>> Forgot to add, broker url only has one query param....
>>
>> jms.prefetchPolicy.queuePrefetch=1
>>
>> which, as I mentioned above, does seem to work.
>>
>>
>> On Tue, Nov 5, 2013 at 10:56 AM, Ned Wolpert <ned.wolpert@imemories.com>wrote:
>>
>>> I can see the preFetch values being set in the console, and they are all
>>> one. I've not set priorities.
>>>
>>> These are 'java' processes, using groovy/grails. The same executable on 4
>>> boxes, each executable with 4 listeners, treaded. Using the grails jms
>>> plugin, which wraps the Spring jms template configuration.
>>> (concurrentConsumers is set to 4 per instance)
>>>
>>> When I have 1000's of messages pending, all instances are working. This
>>> issue is only really viewable when there is 10 messages working.
>>>
>>> The following is the (redacted) activemq.xml.  I'm assuming this config
>>> could be better.  I should mention typical usage of our JMS server has a
>>> few consumers and tons of producers. Thirty queues. Most queues process
>>> quickly and do not fill up. Two queues are for slow producers. The goal is
>>> for the producers to send a message and break away, so we don't want slow
>>> producers at all. Producers are very spiky.... from 10m/min to bursts of
>>> 100's/min.  We have growth concern as that number is increasing steadily.
>>>
>>> <beans
>>>   xmlns="http://www.springframework.org/schema/beans"
>>>   xmlns:amq="http://activemq.apache.org/schema/core"
>>>   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>   xsi:schemaLocation="http://www.springframework.org/schema/beans
>>> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
>>>   http://activemq.apache.org/schema/core
>>> http://activemq.apache.org/schema/core/activemq-core.xsd">
>>>
>>>   <bean
>>> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
>>>     <property name="locations">
>>>       <value>file:${activemq.base}/conf/credentials.properties</value>
>>>     </property>
>>>   </bean>
>>>
>>>   <broker xmlns="http://activemq.apache.org/schema/core"
>>>             brokerName="stagingMQ"
>>>             useJmx="true"
>>>             enableStatistics="true"
>>>             useLocalHostBrokerName="false"
>>>             useLoggingForShutdownErrors="true"
>>>             dataDirectory="XXXXX">
>>>
>>>         <managementContext>
>>>             <managementContext createConnector="true"
>>> connectorPort="XXXXX"/>
>>>         </managementContext>
>>>
>>>         <persistenceAdapter>
>>>            <journaledJDBC journalLogFiles="5"
>>>                           journalLogFileSize="20 Mb"
>>>   dataDirectory="XXXXXX"
>>>                           createTablesOnStartup="false"
>>>                           useDatabaseLock="false"
>>>                           dataSource="#XXXXX">
>>>            </journaledJDBC>
>>>         </persistenceAdapter>
>>>
>>>         <destinationPolicy>
>>>             <policyMap>
>>>               <policyEntries>
>>>                 <policyEntry topic=">" producerFlowControl="true"
>>> memoryLimit="1mb">
>>>                    <pendingSubscriberPolicy>
>>>                     <vmCursor />
>>>                   </pendingSubscriberPolicy>
>>>                 </policyEntry>
>>> <policyEntry queue=">" producerFlowControl="true" memoryLimit="30mb">
>>>                   <pendingQueuePolicy>
>>>                     <vmQueueCursor/>
>>>                   </pendingQueuePolicy>
>>>                 </policyEntry>
>>>               </policyEntries>
>>>             </policyMap>
>>>         </destinationPolicy>
>>>
>>>         <transportConnectors>
>>>             <transportConnector name="openwire" uri="XXXX"/>
>>>             <transportConnector name="stomp" uri="XXXXX"/>
>>>         </transportConnectors>
>>>
>>>     </broker>
>>>
>>>     <import resource="jetty.xml"/>
>>>     <import resource="databaseconfig.xml"/>
>>> </beans>
>>>
>>>
>>>
>>>
>>> On Tue, Nov 5, 2013 at 9:27 AM, Paul Gale <paul.n.gale@gmail.com> wrote:
>>>
>>>> Have you verified via broker logging that the prefetch values you've
>>>> configured are being honored by the broker? Are consumer priorities in
>>>> use? Are your consumers instances of the same executable or are they
>>>> implemented individually?
>>>>
>>>> Can you post your broker configuration: activemq.xml?
>>>>
>>>> How are your clients implemented, e.g., technology: Ruby or Java etc,
>>>> choice of client libraries? Just wondering.
>>>>
>>>>
>>>> Thanks,
>>>> Paul
>>>>
>>>> On Tue, Nov 5, 2013 at 10:28 AM, Ned Wolpert <ned.wolpert@imemories.com>
>>>> wrote:
>>>> > Thanks for the response...
>>>> >
>>>> > Any idea on the round-robin not working? I have a queue with 16
>>>> consumers,
>>>> > all have pre-fetch set to 1. Five consumers are actively processing
>>>> > requests and 3 requests are pending.... the 11 other consumers are
>>>> idle.
>>>> > History has shown that a new request may go to one of the 11 idle
>>>> works,
>>>> > but its like those 3 requests are reserved for some of the working
>>>> ones. I
>>>> > can't figure out what setting would help this, or if this just was a
>>>> bug
>>>> > with 5.3....
>>>> >
>>>> >
>>>> > On Mon, Nov 4, 2013 at 4:37 PM, Christian Posta
>>>> > <christian.posta@gmail.com>wrote:
>>>> >
>>>> >> The clients should negotiate the correct open-wire (protocol version)
>>>> >> so in theory the broker will be backward compatible with older
>>>> >> clients. Just make sure the activemq-openwire-legacy jar is on the
>>>> >> classpath (should be by default).
>>>> >>
>>>> >> Of course I would test this out to make sure :)
>>>> >>
>>>> >> On Mon, Nov 4, 2013 at 10:20 AM, Ned Wolpert <
>>>> ned.wolpert@imemories.com>
>>>> >> wrote:
>>>> >> > Folks-
>>>> >> >
>>>> >> >   I have a 5.3 installation that we're using, and I have 2
>>>> questions for
>>>> >> it:
>>>> >> >
>>>> >> > 1) We have prefetch set to 1 for all of the message consumers
on one
>>>> >> queue,
>>>> >> > where message handling is slow. But it still seems like messages
>>>> aren't
>>>> >> > really 'round robin' to the next available message consumer.
I'll
>>>> see a
>>>> >> few
>>>> >> > consumers are free but messages are waiting around. Is there
a
>>>> >> > configuration that can help?  (I should note that the server
has
>>>> been
>>>> >> > running consistently for 9 months and it seems to be getting
>>>> worse....
>>>> >> > would a restart help?)
>>>> >> >
>>>> >> > 2) We are looking to upgrade to 5.9. I haven't started the
process
>>>> of
>>>> >> > testing, but I wanted to see if this is a case where the 5.3
>>>> clients need
>>>> >> > to be upgraded at the same time as the server, or if the clients
>>>> can be
>>>> >> > rolled over a few weeks to 5.9 after the server gets updated?
>>>> >> >
>>>> >> > Thanks!
>>>> >> >
>>>> >> > --
>>>> >> > Virtually, Ned Wolpert
>>>> >> >
>>>> >> > "Settle thy studies, Faustus, and begin..."   --Marlowe
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Christian Posta
>>>> >> http://www.christianposta.com/blog
>>>> >> twitter: @christianposta
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Virtually, Ned Wolpert
>>>> >
>>>> > "Settle thy studies, Faustus, and begin..."   --Marlowe
>>>>
>>>
>>>
>>>
>>> --
>>> Virtually, Ned Wolpert
>>>
>>> "Settle thy studies, Faustus, and begin..."   --Marlowe
>>>
>>
>>
>>
>> --
>> Virtually, Ned Wolpert
>>
>> "Settle thy studies, Faustus, and begin..."   --Marlowe
>>
>
>
>
> --
> Virtually, Ned Wolpert
>
> "Settle thy studies, Faustus, and begin..."   --Marlowe



-- 
http://redhat.com
http://blog.garytully.com

Mime
View raw message