activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clebert Suconic <clebert.suco...@gmail.com>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Fri, 12 May 2017 19:54:22 GMT
Can you cleanup your data (as a test only?) rm -rf data.




perhaps the spikes you have is the compacting running?

On Fri, May 12, 2017 at 3:51 PM, Michael André Pearce
<michael.andre.pearce@me.com> wrote:
> All I'm doing is changing Artemis.profile of the instance to the different versions.
>
>
> Sent from my iPhone
>
>> On 12 May 2017, at 20:35, nigro_franz <nigro.fra@gmail.com> wrote:
>>
>> Too strange...have you tried with NIO too?It uses always the same buffer to
>> batch writes AFAIK....
>>
>> 2017-05-12 21:30 GMT+02:00 Michael André Pearce [via ActiveMQ] <
>> ml+s2283324n4726091h96@n4.nabble.com>:
>>
>>> Ok so it isn't the times buffer changes. Just deployed 2.1.0 with and
>>> without the timed buffer changes. Running the same tests no difference.
>>>
>>> Must be something else. Causing the behaviour change.
>>>
>>> Sent from my iPhone
>>>
>>>> On 12 May 2017, at 18:10, Michael André Pearce <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>> wrote:
>>>>
>>>> I've left for the day now. If I get time over the weekend I'll try see
>>> if I can make a build of 2.1.0 without that change and see if it makes any
>>> difference.
>>>>
>>>> Can someone point me to the PR for that change, so I know what I'm
>>> unpicking locally?
>>>>
>>>> Sent from my iPhone
>>>>
>>>>> On 12 May 2017, at 17:58, Clebert Suconic <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>> wrote:
>>>>>
>>>>> As the only thing that could affect this is the Change on timed buffer.
>>>>> Afaik
>>>>>
>>>>>
>>>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>>
>>>>> wrote:
>>>>>
>>>>>> I'm considering only keeping the pooled buffer part and switch back
to
>>> the
>>>>>> ole sleep or an improved sleep we had.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce <
>>>>>> [hidden email] <http:///user/SendEmail.jtp?type=node&node=4726091&i=3>>
>>> wrote:
>>>>>>
>>>>>>> As it seems  I can't send images to mail list, just sent to 
you both
>>> via
>>>>>>> email. Some graphs we have comparing versions.
>>>>>>>
>>>>>>> Not sure what changes might cause it.
>>>>>>>
>>>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> There is a class we use on producer.  TokenLimiter.  Perhaps
you
>>> could
>>>>>>>> reuse that one ?
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <[hidden
email]
>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I was thinking of a similar solution but I've discovered
that
>>> couldn't
>>>>>>> work
>>>>>>>>> (in the old or the new TimedBuffer too), because of the
>>>>>>>>> TimedBuffer::checkSize method that could force a flush
if the batch
>>>>>>> buffer
>>>>>>>>> if not big enough to receive new data, going IOPS.
>>>>>>>>> Sadly TimedBuffer::checkSize is outside any timeout,
but depends on
>>> the
>>>>>>>>> writers.
>>>>>>>>>
>>>>>>>>> That's why I've implemented the "compensation" right
after any
>>> flush,
>>>>>>> in
>>>>>>>>> order to work with forced flushes too:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> https://github.com/franz1981/activemq-artemis/blob/
>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
>>> src/main/java/org/apache/activemq/artemis/core/io/
>>> buffer/TimedBuffer.java#L338
>>>>>>>>>
>>>>>>>>> Doing it on TimedBuffer::flush all the flushes on disk
will be
>>>>>>> compensated
>>>>>>>>> (half of the story: ASYNCIO is async so depends on libAIO
>>> partially!)
>>>>>>>>> Regarding the IOPS computation I've built this, as you've
>>> suggested:
>>>>>>>>>
>>>>>>>>>
>>>>>>> https://github.com/franz1981/activemq-artemis/blob/
>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
>>> src/main/java/org/apache/activemq/artemis/core/io/
>>> buffer/TimedBuffer.java#L119
>>>>>>>>>
>>>>>>>>> The performance seems pretty good, it compensates well
but it is
>>> faster
>>>>>>>>> than
>>>>>>>>> the original version, limiting IOPS too!
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Franz
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context:
>>>>>>>>>
>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>>> IOPS-Limiter-strategy-tp4725875p4726057.html
>>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive at
Nabble.com.
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Clebert Suconic
>>>>>>>
>>>>>> --
>>>>>> Clebert Suconic
>>>>>>
>>>>> --
>>>>> Clebert Suconic
>>>
>>>
>>> ------------------------------
>>> If you reply to this email, your message will be added to the discussion
>>> below:
>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>>> IOPS-Limiter-strategy-tp4725875p4726091.html
>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here
>>> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0NzI1ODc1fC0zNzI3NjE2NQ==>
>>> .
>>> NAML
>>> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>
>>
>>
>>
>> --
>> View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726093.html
>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.



-- 
Clebert Suconic

Mime
View raw message