activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael André Pearce <michael.andre.pea...@me.com>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Fri, 12 May 2017 20:31:46 GMT
Yup deleting the data on version change removes the issue.

Sent from my iPhone

> On 12 May 2017, at 20:54, Clebert Suconic <clebert.suconic@gmail.com> wrote:
> 
> Can you cleanup your data (as a test only?) rm -rf data.
> 
> 
> 
> 
> perhaps the spikes you have is the compacting running?
> 
> On Fri, May 12, 2017 at 3:51 PM, Michael André Pearce
> <michael.andre.pearce@me.com> wrote:
>> All I'm doing is changing Artemis.profile of the instance to the different versions.
>> 
>> 
>> Sent from my iPhone
>> 
>>> On 12 May 2017, at 20:35, nigro_franz <nigro.fra@gmail.com> wrote:
>>> 
>>> Too strange...have you tried with NIO too?It uses always the same buffer to
>>> batch writes AFAIK....
>>> 
>>> 2017-05-12 21:30 GMT+02:00 Michael André Pearce [via ActiveMQ] <
>>> ml+s2283324n4726091h96@n4.nabble.com>:
>>> 
>>>> Ok so it isn't the times buffer changes. Just deployed 2.1.0 with and
>>>> without the timed buffer changes. Running the same tests no difference.
>>>> 
>>>> Must be something else. Causing the behaviour change.
>>>> 
>>>> Sent from my iPhone
>>>> 
>>>>> On 12 May 2017, at 18:10, Michael André Pearce <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>>
wrote:
>>>>> 
>>>>> I've left for the day now. If I get time over the weekend I'll try see
>>>> if I can make a build of 2.1.0 without that change and see if it makes any
>>>> difference.
>>>>> 
>>>>> Can someone point me to the PR for that change, so I know what I'm
>>>> unpicking locally?
>>>>> 
>>>>> Sent from my iPhone
>>>>> 
>>>>>> On 12 May 2017, at 17:58, Clebert Suconic <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>>
wrote:
>>>>>> 
>>>>>> As the only thing that could affect this is the Change on timed buffer.
>>>>>> Afaik
>>>>>> 
>>>>>> 
>>>>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>>
>>>>>> wrote:
>>>>>> 
>>>>>>> I'm considering only keeping the pooled buffer part and switch
back to
>>>> the
>>>>>>> ole sleep or an improved sleep we had.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce <
>>>>>>> [hidden email] <http:///user/SendEmail.jtp?type=node&node=4726091&i=3>>
>>>> wrote:
>>>>>>> 
>>>>>>>> As it seems  I can't send images to mail list, just sent
to  you both
>>>> via
>>>>>>>> email. Some graphs we have comparing versions.
>>>>>>>> 
>>>>>>>> Not sure what changes might cause it.
>>>>>>>> 
>>>>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <[hidden
email]
>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> There is a class we use on producer.  TokenLimiter. 
Perhaps you
>>>> could
>>>>>>>>> reuse that one ?
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <[hidden
email]
>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>>
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> I was thinking of a similar solution but I've discovered
that
>>>> couldn't
>>>>>>>> work
>>>>>>>>>> (in the old or the new TimedBuffer too), because
of the
>>>>>>>>>> TimedBuffer::checkSize method that could force a
flush if the batch
>>>>>>>> buffer
>>>>>>>>>> if not big enough to receive new data, going IOPS.
>>>>>>>>>> Sadly TimedBuffer::checkSize is outside any timeout,
but depends on
>>>> the
>>>>>>>>>> writers.
>>>>>>>>>> 
>>>>>>>>>> That's why I've implemented the "compensation" right
after any
>>>> flush,
>>>>>>>> in
>>>>>>>>>> order to work with forced flushes too:
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/
>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
>>>> src/main/java/org/apache/activemq/artemis/core/io/
>>>> buffer/TimedBuffer.java#L338
>>>>>>>>>> 
>>>>>>>>>> Doing it on TimedBuffer::flush all the flushes on
disk will be
>>>>>>>> compensated
>>>>>>>>>> (half of the story: ASYNCIO is async so depends on
libAIO
>>>> partially!)
>>>>>>>>>> Regarding the IOPS computation I've built this, as
you've
>>>> suggested:
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/
>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
>>>> src/main/java/org/apache/activemq/artemis/core/io/
>>>> buffer/TimedBuffer.java#L119
>>>>>>>>>> 
>>>>>>>>>> The performance seems pretty good, it compensates
well but it is
>>>> faster
>>>>>>>>>> than
>>>>>>>>>> the original version, limiting IOPS too!
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> Franz
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> 
>>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>>>> IOPS-Limiter-strategy-tp4725875p4726057.html
>>>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive
at Nabble.com.
>>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Clebert Suconic
>>>>>>>> 
>>>>>>> --
>>>>>>> Clebert Suconic
>>>>>>> 
>>>>>> --
>>>>>> Clebert Suconic
>>>> 
>>>> 
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the discussion
>>>> below:
>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>>>> IOPS-Limiter-strategy-tp4725875p4726091.html
>>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here
>>>> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0NzI1ODc1fC0zNzI3NjE2NQ==>
>>>> .
>>>> NAML
>>>> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726093.html
>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
> 
> 
> 
> -- 
> Clebert Suconic

Mime
View raw message