activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael André Pearce <michael.andre.pea...@me.com>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Fri, 12 May 2017 19:51:53 GMT
All I'm doing is changing Artemis.profile of the instance to the different versions. 


Sent from my iPhone

> On 12 May 2017, at 20:35, nigro_franz <nigro.fra@gmail.com> wrote:
> 
> Too strange...have you tried with NIO too?It uses always the same buffer to
> batch writes AFAIK....
> 
> 2017-05-12 21:30 GMT+02:00 Michael André Pearce [via ActiveMQ] <
> ml+s2283324n4726091h96@n4.nabble.com>:
> 
>> Ok so it isn't the times buffer changes. Just deployed 2.1.0 with and
>> without the timed buffer changes. Running the same tests no difference.
>> 
>> Must be something else. Causing the behaviour change.
>> 
>> Sent from my iPhone
>> 
>>> On 12 May 2017, at 18:10, Michael André Pearce <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>> wrote:
>>> 
>>> I've left for the day now. If I get time over the weekend I'll try see
>> if I can make a build of 2.1.0 without that change and see if it makes any
>> difference.
>>> 
>>> Can someone point me to the PR for that change, so I know what I'm
>> unpicking locally?
>>> 
>>> Sent from my iPhone
>>> 
>>>> On 12 May 2017, at 17:58, Clebert Suconic <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>> wrote:
>>>> 
>>>> As the only thing that could affect this is the Change on timed buffer.
>>>> Afaik
>>>> 
>>>> 
>>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>>
>>>> wrote:
>>>> 
>>>>> I'm considering only keeping the pooled buffer part and switch back to
>> the
>>>>> ole sleep or an improved sleep we had.
>>>>> 
>>>>> 
>>>>> 
>>>>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce <
>>>>> [hidden email] <http:///user/SendEmail.jtp?type=node&node=4726091&i=3>>
>> wrote:
>>>>> 
>>>>>> As it seems  I can't send images to mail list, just sent to  you
both
>> via
>>>>>> email. Some graphs we have comparing versions.
>>>>>> 
>>>>>> Not sure what changes might cause it.
>>>>>> 
>>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>>
>>>>>> wrote:
>>>>>>> 
>>>>>>> There is a class we use on producer.  TokenLimiter.  Perhaps
you
>> could
>>>>>>> reuse that one ?
>>>>>>> 
>>>>>>> 
>>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <[hidden
email]
>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>>
>>>>>> wrote:
>>>>>>>> 
>>>>>>>> I was thinking of a similar solution but I've discovered
that
>> couldn't
>>>>>> work
>>>>>>>> (in the old or the new TimedBuffer too), because of the
>>>>>>>> TimedBuffer::checkSize method that could force a flush if
the batch
>>>>>> buffer
>>>>>>>> if not big enough to receive new data, going IOPS.
>>>>>>>> Sadly TimedBuffer::checkSize is outside any timeout, but
depends on
>> the
>>>>>>>> writers.
>>>>>>>> 
>>>>>>>> That's why I've implemented the "compensation" right after
any
>> flush,
>>>>>> in
>>>>>>>> order to work with forced flushes too:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>> https://github.com/franz1981/activemq-artemis/blob/
>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
>> src/main/java/org/apache/activemq/artemis/core/io/
>> buffer/TimedBuffer.java#L338
>>>>>>>> 
>>>>>>>> Doing it on TimedBuffer::flush all the flushes on disk will
be
>>>>>> compensated
>>>>>>>> (half of the story: ASYNCIO is async so depends on libAIO
>> partially!)
>>>>>>>> Regarding the IOPS computation I've built this, as you've
>> suggested:
>>>>>>>> 
>>>>>>>> 
>>>>>> https://github.com/franz1981/activemq-artemis/blob/
>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
>> src/main/java/org/apache/activemq/artemis/core/io/
>> buffer/TimedBuffer.java#L119
>>>>>>>> 
>>>>>>>> The performance seems pretty good, it compensates well but
it is
>> faster
>>>>>>>> than
>>>>>>>> the original version, limiting IOPS too!
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Franz
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>> 
>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>> IOPS-Limiter-strategy-tp4725875p4726057.html
>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>>>>>>> 
>>>>>>> --
>>>>>>> Clebert Suconic
>>>>>> 
>>>>> --
>>>>> Clebert Suconic
>>>>> 
>>>> --
>>>> Clebert Suconic
>> 
>> 
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>> IOPS-Limiter-strategy-tp4725875p4726091.html
>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here
>> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0NzI1ODc1fC0zNzI3NjE2NQ==>
>> .
>> NAML
>> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>> 
> 
> 
> 
> 
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726093.html
> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.

Mime
View raw message