activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Davies <rajdav...@gmail.com>
Subject Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0 releases
Date Tue, 22 Sep 2009 11:59:25 GMT
Thats the problem - persistent Queue with no subscribers shouldn't  
block until store limit is reached - this is why flow control has been  
disabled by default now
On 21 Sep 2009, at 23:29, Colin MacNaughton wrote:

> Hi Rob,
>
> I didn't run such a test, but I'd expect that the queue would pretty  
> quickly
> fill up and block the senders using the config snippet below since  
> it limits
> the queue size to 1Mb.
>
> Colin
>
> -----Original Message-----
> From: Rob Davies [mailto:rajdavies@gmail.com]
> Sent: Monday, September 21, 2009 3:25 PM
> To: dev@activemq.apache.org
> Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ  
> 5.3.0
> releases
>
> Hey Colin - what results do you see with flow control on and no
> consumers for persistent queues ?
>
> On 21 Sep 2009, at 21:01, Colin MacNaughton wrote:
>
>> So ran into 2 issues running performance tests:
>>
>> 1. I ended up tweaking the default config to limit destination sizes
>> and
>> enable flow control as follows:
>>
>> <destinationPolicy>
>> <policyMap>
>>  <policyEntries>
>>    <policyEntry topic=">" producerFlowControl="true"
>> memoryLimit="1mb">
>>
>>      <pendingSubscriberPolicy>
>>        <vmCursor/>
>>      </pendingSubscriberPolicy>
>>    </policyEntry>
>>    <policyEntry queue=">" producerFlowControl="true"
>> memoryLimit="1mb"/>
>>  </policyEntries>
>> </policyMap>
>> </destinationPolicy>
>>
>> The current default config was resulting in really high latencies in
>> non
>> persistent pub sub tests (> 2 minutes!). With the new settings
>> throughput
>> doubled and average latency dropped to 3 seconds.
>>
>> However, it seems like there is some resistance to enabling flow
>> control by
>> default: http://issues.apache.org/activemq/browse/AMQ-2318, as naïve
>> users
>> might erroneously interpret this as a hang.
>>
>> So there is a tradeoff here against guarding again naïve users and
>> good out
>> of box performance benchmarking.
>>
>> A possible compromise appropriate for the 5.3.0 release time frame
>> would be
>> to log a warning the first time flow control is triggered for a
>> destination,
>> to assist naive users in troubleshooting producer pauses.
>>
>> More long term, it might be worth introucing a more sophisticated
>> mechanism
>> for when we page to disk like only do so when there are no consumers
>> connected. A policy similar to this is already being pursued in the
>> amq 6.0
>> prototype.
>>
>> I logged this as http://issues.apache.org/activemq/browse/AMQ-2400
>>
>> 2. Fan-in to dups_ok queue receivers:
>> While running performance tests I I was seeing hangs in several tests
>> involving dups ok queue receivers. My suspicion is that this is
>> related to
>> "too lazy" dups_ok acknowledgements. Changing the queue
>> prefetchLimit to 100
>> caused this problem to go away. This needs more investigation, but
>> it seems
>> like we can get ourselves in to trouble if the queue size is smaller
>> than
>> the receiver's prefetchLimit, and this should be avoid. It is also
>> possible
>> that there is something more complicated happening in my tests. I
>> haven't
>> yet been able to reproduce this outside my performance test
>> environment.
>>
>> Logged as http://issues.apache.org/activemq/browse/AMQ-2401
>>
>> -----Original Message-----
>> From: Colin MacNaughton [mailto:colin.macnaughton@gmail.com]
>> Sent: Friday, September 18, 2009 12:49 PM
>> To: dev@activemq.apache.org
>> Subject: RE: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
>> 5.3.0
>> releases
>>
>> Hey Dejan,
>>
>> FYI, I'm running the RC though the Progress internal performance
>> test suite
>> over the weekend. Will advise of the results, but it should be
>> interesting
>> to see how the new default config performs, and we can see if we
>> need to
>> tweak it.
>>
>> Colin
>>
>> -----Original Message-----
>> From: chubrilo@gmail.com [mailto:chubrilo@gmail.com] On Behalf Of
>> Dejan
>> Bosanac
>> Sent: Friday, September 18, 2009 11:14 AM
>> To: dev@activemq.apache.org
>> Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
>> 5.3.0
>> releases
>>
>> Ok, I'll modify tomorrow how we create source release and include
>> protobuf
>> code in it. I guess I'll need to tweak assembly-plugin and
>> apache-source-release-assembly-descriptor, but have to research it
>> more on
>> how to do it. If anybody has any experience with this and would
>> provide any
>> pointers it would be very helpful.
>>
>> Cheers
>> --
>> Dejan Bosanac
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
>>
>> On Fri, Sep 18, 2009 at 7:02 PM, Hiram Chirino <chirino@gmail.com>
>> wrote:
>>
>>> Yeah that that does not have the source tar ball for the protobuf
>>> release.
>>>
>>> On Fri, Sep 18, 2009 at 12:52 PM, Bruce Snyder
>>> <bruce.snyder@gmail.com
>>>> wrote:
>>>
>>>> On Fri, Sep 18, 2009 at 7:10 AM, Hiram Chirino <chirino@gmail.com>
>>> wrote:
>>>>> Could you also post links to the source tarballs?  Thanks!
>>>>
>>>> He already did:
>>>>
>>>>
>>>>
>>>
>>
> https://repository.apache.org/content/repositories/activemq-staging-030/org/
>> apache/activemq/activemq-parent/5.3.0/
>>>>
>>>> Bruce
>>>> --
>>>> perl -e 'print
>>>> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I; 
>>>> \"YC;VT*"
>>>> );'
>>>>
>>>> ActiveMQ in Action: http://bit.ly/2je6cQ
>>>> Blog: http://bruceblog.org/
>>>> Twitter: http://twitter.com/brucesnyder
>>>>
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Hiram
>>>
>>> Blog: http://hiramchirino.com
>>>
>>> Open Source SOA
>>> http://fusesource.com/
>>>
>>
>>
>
> Rob Davies
> http://twitter.com/rajdavies
> I work here: http://fusesource.com
> My Blog: http://rajdavies.blogspot.com/
> I'm writing this: http://www.manning.com/snyder/
>
>
>
>
>
>

Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/






Mime
View raw message