activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ned Wolpert <ned.wolp...@imemories.com>
Subject Re: 5.3 question and server upgrade question...
Date Wed, 06 Nov 2013 16:40:37 GMT
Forgot to add, broker url only has one query param....

jms.prefetchPolicy.queuePrefetch=1

which, as I mentioned above, does seem to work.


On Tue, Nov 5, 2013 at 10:56 AM, Ned Wolpert <ned.wolpert@imemories.com>wrote:

> I can see the preFetch values being set in the console, and they are all
> one. I've not set priorities.
>
> These are 'java' processes, using groovy/grails. The same executable on 4
> boxes, each executable with 4 listeners, treaded. Using the grails jms
> plugin, which wraps the Spring jms template configuration.
> (concurrentConsumers is set to 4 per instance)
>
> When I have 1000's of messages pending, all instances are working. This
> issue is only really viewable when there is 10 messages working.
>
> The following is the (redacted) activemq.xml.  I'm assuming this config
> could be better.  I should mention typical usage of our JMS server has a
> few consumers and tons of producers. Thirty queues. Most queues process
> quickly and do not fill up. Two queues are for slow producers. The goal is
> for the producers to send a message and break away, so we don't want slow
> producers at all. Producers are very spiky.... from 10m/min to bursts of
> 100's/min.  We have growth concern as that number is increasing steadily.
>
> <beans
>   xmlns="http://www.springframework.org/schema/beans"
>   xmlns:amq="http://activemq.apache.org/schema/core"
>   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>   xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
>   http://activemq.apache.org/schema/core
> http://activemq.apache.org/schema/core/activemq-core.xsd">
>
>   <bean
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
>     <property name="locations">
>       <value>file:${activemq.base}/conf/credentials.properties</value>
>     </property>
>   </bean>
>
>   <broker xmlns="http://activemq.apache.org/schema/core"
>             brokerName="stagingMQ"
>             useJmx="true"
>             enableStatistics="true"
>             useLocalHostBrokerName="false"
>             useLoggingForShutdownErrors="true"
>             dataDirectory="XXXXX">
>
>         <managementContext>
>             <managementContext createConnector="true"
> connectorPort="XXXXX"/>
>         </managementContext>
>
>         <persistenceAdapter>
>            <journaledJDBC journalLogFiles="5"
>                           journalLogFileSize="20 Mb"
>   dataDirectory="XXXXXX"
>                           createTablesOnStartup="false"
>                           useDatabaseLock="false"
>                           dataSource="#XXXXX">
>            </journaledJDBC>
>         </persistenceAdapter>
>
>         <destinationPolicy>
>             <policyMap>
>               <policyEntries>
>                 <policyEntry topic=">" producerFlowControl="true"
> memoryLimit="1mb">
>                   <pendingSubscriberPolicy>
>                     <vmCursor />
>                   </pendingSubscriberPolicy>
>                 </policyEntry>
> <policyEntry queue=">" producerFlowControl="true" memoryLimit="30mb">
>                   <pendingQueuePolicy>
>                     <vmQueueCursor/>
>                   </pendingQueuePolicy>
>                 </policyEntry>
>               </policyEntries>
>             </policyMap>
>         </destinationPolicy>
>
>         <transportConnectors>
>             <transportConnector name="openwire" uri="XXXX"/>
>             <transportConnector name="stomp" uri="XXXXX"/>
>         </transportConnectors>
>
>     </broker>
>
>     <import resource="jetty.xml"/>
>     <import resource="databaseconfig.xml"/>
> </beans>
>
>
>
>
> On Tue, Nov 5, 2013 at 9:27 AM, Paul Gale <paul.n.gale@gmail.com> wrote:
>
>> Have you verified via broker logging that the prefetch values you've
>> configured are being honored by the broker? Are consumer priorities in
>> use? Are your consumers instances of the same executable or are they
>> implemented individually?
>>
>> Can you post your broker configuration: activemq.xml?
>>
>> How are your clients implemented, e.g., technology: Ruby or Java etc,
>> choice of client libraries? Just wondering.
>>
>>
>> Thanks,
>> Paul
>>
>> On Tue, Nov 5, 2013 at 10:28 AM, Ned Wolpert <ned.wolpert@imemories.com>
>> wrote:
>> > Thanks for the response...
>> >
>> > Any idea on the round-robin not working? I have a queue with 16
>> consumers,
>> > all have pre-fetch set to 1. Five consumers are actively processing
>> > requests and 3 requests are pending.... the 11 other consumers are idle.
>> > History has shown that a new request may go to one of the 11 idle works,
>> > but its like those 3 requests are reserved for some of the working
>> ones. I
>> > can't figure out what setting would help this, or if this just was a bug
>> > with 5.3....
>> >
>> >
>> > On Mon, Nov 4, 2013 at 4:37 PM, Christian Posta
>> > <christian.posta@gmail.com>wrote:
>> >
>> >> The clients should negotiate the correct open-wire (protocol version)
>> >> so in theory the broker will be backward compatible with older
>> >> clients. Just make sure the activemq-openwire-legacy jar is on the
>> >> classpath (should be by default).
>> >>
>> >> Of course I would test this out to make sure :)
>> >>
>> >> On Mon, Nov 4, 2013 at 10:20 AM, Ned Wolpert <
>> ned.wolpert@imemories.com>
>> >> wrote:
>> >> > Folks-
>> >> >
>> >> >   I have a 5.3 installation that we're using, and I have 2 questions
>> for
>> >> it:
>> >> >
>> >> > 1) We have prefetch set to 1 for all of the message consumers on one
>> >> queue,
>> >> > where message handling is slow. But it still seems like messages
>> aren't
>> >> > really 'round robin' to the next available message consumer. I'll
>> see a
>> >> few
>> >> > consumers are free but messages are waiting around. Is there a
>> >> > configuration that can help?  (I should note that the server has been
>> >> > running consistently for 9 months and it seems to be getting
>> worse....
>> >> > would a restart help?)
>> >> >
>> >> > 2) We are looking to upgrade to 5.9. I haven't started the process
of
>> >> > testing, but I wanted to see if this is a case where the 5.3 clients
>> need
>> >> > to be upgraded at the same time as the server, or if the clients can
>> be
>> >> > rolled over a few weeks to 5.9 after the server gets updated?
>> >> >
>> >> > Thanks!
>> >> >
>> >> > --
>> >> > Virtually, Ned Wolpert
>> >> >
>> >> > "Settle thy studies, Faustus, and begin..."   --Marlowe
>> >>
>> >>
>> >>
>> >> --
>> >> Christian Posta
>> >> http://www.christianposta.com/blog
>> >> twitter: @christianposta
>> >>
>> >
>> >
>> >
>> > --
>> > Virtually, Ned Wolpert
>> >
>> > "Settle thy studies, Faustus, and begin..."   --Marlowe
>>
>
>
>
> --
> Virtually, Ned Wolpert
>
> "Settle thy studies, Faustus, and begin..."   --Marlowe
>



-- 
Virtually, Ned Wolpert

"Settle thy studies, Faustus, and begin..."   --Marlowe

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message