activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael André Pearce <michael.andre.pea...@me.com>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Fri, 12 May 2017 17:10:06 GMT
I've left for the day now. If I get time over the weekend I'll try see if I can make a build
of 2.1.0 without that change and see if it makes any difference.

Can someone point me to the PR for that change, so I know what I'm unpicking locally?

Sent from my iPhone

> On 12 May 2017, at 17:58, Clebert Suconic <clebert.suconic@gmail.com> wrote:
> 
> As the only thing that could affect this is the Change on timed buffer.
> Afaik
> 
> 
> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <clebert.suconic@gmail.com>
> wrote:
> 
>> I'm considering only keeping the pooled buffer part and switch back to the
>> ole sleep or an improved sleep we had.
>> 
>> 
>> 
>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce <
>> michael.andre.pearce@me.com> wrote:
>> 
>>> As it seems  I can't send images to mail list, just sent to  you both via
>>> email. Some graphs we have comparing versions.
>>> 
>>> Not sure what changes might cause it.
>>> 
>>>> On 12 May 2017, at 17:37, Clebert Suconic <clebert.suconic@gmail.com>
>>> wrote:
>>>> 
>>>> There is a class we use on producer.  TokenLimiter.  Perhaps you could
>>>> reuse that one ?
>>>> 
>>>> 
>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <nigro.fra@gmail.com>
>>> wrote:
>>>>> 
>>>>> I was thinking of a similar solution but I've discovered that couldn't
>>> work
>>>>> (in the old or the new TimedBuffer too), because of the
>>>>> TimedBuffer::checkSize method that could force a flush if the batch
>>> buffer
>>>>> if not big enough to receive new data, going IOPS.
>>>>> Sadly TimedBuffer::checkSize is outside any timeout, but depends on the
>>>>> writers.
>>>>> 
>>>>> That's why I've implemented the "compensation" right after any flush,
>>> in
>>>>> order to work with forced flushes too:
>>>>> 
>>>>> 
>>>>> 
>>> https://github.com/franz1981/activemq-artemis/blob/4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java#L338
>>>>> 
>>>>> Doing it on TimedBuffer::flush all the flushes on disk will be
>>> compensated
>>>>> (half of the story: ASYNCIO is async so depends on libAIO partially!)
>>>>> Regarding the IOPS computation I've built this, as you've suggested:
>>>>> 
>>>>> 
>>> https://github.com/franz1981/activemq-artemis/blob/4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java#L119
>>>>> 
>>>>> The performance seems pretty good, it compensates well but it is faster
>>>>> than
>>>>> the original version, limiting IOPS too!
>>>>> 
>>>>> Thanks,
>>>>> Franz
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> View this message in context:
>>>>> 
>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726057.html
>>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>>>> 
>>>> --
>>>> Clebert Suconic
>>> 
>> --
>> Clebert Suconic
>> 
> -- 
> Clebert Suconic

Mime
View raw message