activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Bertram <jbert...@apache.org>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Fri, 12 May 2017 17:03:07 GMT
> As it seems I can't send images to mail list...

FWIW, you can add images to gist [1] and then send links to the list.  I just tried it myself
and it works as advertised.


Justin

[1] https://gist.github.com/remarkablemark/feff40b0a522f0c41c4eff0b77ea1d47

----- Original Message -----
From: "Michael André Pearce" <michael.andre.pearce@me.com>
To: dev@activemq.apache.org
Sent: Friday, May 12, 2017 11:48:50 AM
Subject: Re: [DISCUSS] Artemis IOPS Limiter strategy

As it seems I can't send images to mail list, just sent to you both via email. Some graphs
we have comparing versions.

Not sure what changes might cause it.

> On 12 May 2017, at 17:37, Clebert Suconic <clebert.suconic@gmail.com> wrote:
> 
> There is a class we use on producer.  TokenLimiter.  Perhaps you could
> reuse that one ?
> 
> 
>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <nigro.fra@gmail.com> wrote:
>> 
>> I was thinking of a similar solution but I've discovered that couldn't work
>> (in the old or the new TimedBuffer too), because of the
>> TimedBuffer::checkSize method that could force a flush if the batch buffer
>> if not big enough to receive new data, going IOPS.
>> Sadly TimedBuffer::checkSize is outside any timeout, but depends on the
>> writers.
>> 
>> That's why I've implemented the "compensation" right after any flush, in
>> order to work with forced flushes too:
>> 
>> 
>> https://github.com/franz1981/activemq-artemis/blob/4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java#L338
>> 
>> Doing it on TimedBuffer::flush all the flushes on disk will be compensated
>> (half of the story: ASYNCIO is async so depends on libAIO partially!)
>> Regarding the IOPS computation I've built this, as you've suggested:
>> 
>> https://github.com/franz1981/activemq-artemis/blob/4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java#L119
>> 
>> The performance seems pretty good, it compensates well but it is faster
>> than
>> the original version, limiting IOPS too!
>> 
>> Thanks,
>> Franz
>> 
>> 
>> 
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726057.html
>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>> 
> -- 
> Clebert Suconic

Mime
View raw message