activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clebert Suconic <clebert.suco...@gmail.com>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Fri, 12 May 2017 19:52:59 GMT
The only difference in IO applied was:

https://issues.apache.org/jira/browse/ARTEMIS-1117



There's also the MaxSize that we now set the maxSize at half the -Xmx
(if you create a new broker).. that may have some implications on your
GC depending on the test you are doing.. perhaps you could use a
global max defined?

On Fri, May 12, 2017 at 3:42 PM, Michael André Pearce
<michael.andre.pearce@me.com> wrote:
> Ok so it isn't the times buffer changes. Just deployed 2.1.0 with and without the timed
buffer changes. Running the same tests no difference.
>
> Must be something else. Causing the behaviour change.
>
> Sent from my iPhone
>
>> On 12 May 2017, at 18:10, Michael André Pearce <michael.andre.pearce@me.com>
wrote:
>>
>> I've left for the day now. If I get time over the weekend I'll try see if I can make
a build of 2.1.0 without that change and see if it makes any difference.
>>
>> Can someone point me to the PR for that change, so I know what I'm unpicking locally?
>>
>> Sent from my iPhone
>>
>>> On 12 May 2017, at 17:58, Clebert Suconic <clebert.suconic@gmail.com> wrote:
>>>
>>> As the only thing that could affect this is the Change on timed buffer.
>>> Afaik
>>>
>>>
>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <clebert.suconic@gmail.com>
>>> wrote:
>>>
>>>> I'm considering only keeping the pooled buffer part and switch back to the
>>>> ole sleep or an improved sleep we had.
>>>>
>>>>
>>>>
>>>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce <
>>>> michael.andre.pearce@me.com> wrote:
>>>>
>>>>> As it seems  I can't send images to mail list, just sent to  you both
via
>>>>> email. Some graphs we have comparing versions.
>>>>>
>>>>> Not sure what changes might cause it.
>>>>>
>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <clebert.suconic@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> There is a class we use on producer.  TokenLimiter.  Perhaps you
could
>>>>>> reuse that one ?
>>>>>>
>>>>>>
>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <nigro.fra@gmail.com>
>>>>> wrote:
>>>>>>>
>>>>>>> I was thinking of a similar solution but I've discovered that
couldn't
>>>>> work
>>>>>>> (in the old or the new TimedBuffer too), because of the
>>>>>>> TimedBuffer::checkSize method that could force a flush if the
batch
>>>>> buffer
>>>>>>> if not big enough to receive new data, going IOPS.
>>>>>>> Sadly TimedBuffer::checkSize is outside any timeout, but depends
on the
>>>>>>> writers.
>>>>>>>
>>>>>>> That's why I've implemented the "compensation" right after any
flush,
>>>>> in
>>>>>>> order to work with forced flushes too:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>> https://github.com/franz1981/activemq-artemis/blob/4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java#L338
>>>>>>>
>>>>>>> Doing it on TimedBuffer::flush all the flushes on disk will be
>>>>> compensated
>>>>>>> (half of the story: ASYNCIO is async so depends on libAIO partially!)
>>>>>>> Regarding the IOPS computation I've built this, as you've suggested:
>>>>>>>
>>>>>>>
>>>>> https://github.com/franz1981/activemq-artemis/blob/4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/src/main/java/org/apache/activemq/artemis/core/io/buffer/TimedBuffer.java#L119
>>>>>>>
>>>>>>> The performance seems pretty good, it compensates well but it
is faster
>>>>>>> than
>>>>>>> the original version, limiting IOPS too!
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Franz
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>>
>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726057.html
>>>>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>>>>>>
>>>>>> --
>>>>>> Clebert Suconic
>>>>>
>>>> --
>>>> Clebert Suconic
>>>>
>>> --
>>> Clebert Suconic



-- 
Clebert Suconic

Mime
View raw message