activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael André Pearce <michael.andre.pea...@me.com>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Sat, 13 May 2017 04:10:34 GMT
My request would be can we make it configurable via broker.xml to have new and old behaviour.

So that 

a) during testing it's easy to switch between the two solutions, instead of having to deploy
different versions :) 

b) existing deployments don't get any unexpected change in behaviour (or at least can control
the change, or revert it if any undesired effects seen)




Sent from my iPhone

> On 13 May 2017, at 01:58, Clebert Suconic <clebert.suconic@gmail.com> wrote:
> 
> With Asyncio you have to flood requests a bit beyond the syncs per second.
> That's why we calculate for aio separately.
> 
> We may need to eventually add more threads to poll.
> 
> But the good thing on aio is using less CPU.
> 
> 
> 
>> On Fri, May 12, 2017 at 5:25 PM Francesco Nigro <nigro.fra@gmail.com> wrote:
>> 
>> And the most weird thing is that I'm having better results with NIO then
>> ASYNCIO (with small sized messages) and I'm not getting why....
>> 
>> Time to go, 'm off; in Italy is 23:30 now..have fun guys and thanks: always
>> great stuff :)
>> 
>> 2017-05-12 23:02 GMT+02:00 nigro_franz <nigro.fra@gmail.com>:
>> 
>>> ahah np Michael!!!
>>> 
>>> Anyway I was really expecting some more spike compared to the old
>>> TimedBuffer: Is one of the reasons why I wanted to put the IO
>> compensation.
>>> The new one seems to work well (with ASYNCIO) in this regard by the tests
>>> I've done: I'm hoping it will help to remove that little spikes too!
>>> 
>>> 
>>> 2017-05-12 22:54 GMT+02:00 Michael André Pearce [via ActiveMQ] <
>>> ml+s2283324n4726101h6@n4.nabble.com>:
>>> 
>>>> Correct, :) seems like tester error. Apologies
>>>> 
>>>> Sent from my iPhone
>>>> 
>>>>> On 12 May 2017, at 21:24, nigro_franz <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=4726101&i=0>>
wrote:
>>>>> 
>>>>> Hence the new TimedBuffer didn't change drastically the latencies?
>>>>> I've got some graph after the stress test with a maxed IO, if you're
>>>>> curious :)
>>>>> 
>>>>> 2017-05-12 22:19 GMT+02:00 Michael André Pearce [via ActiveMQ] <
>>>>> [hidden email] <http:///user/SendEmail.jtp?
>>> type=node&node=4726101&i=1>>:
>>>> 
>>>>> 
>>>>>> Yup deleting the data on version change removes the issue.
>>>>>> 
>>>>>> Sent from my iPhone
>>>>>> 
>>>>>>> On 12 May 2017, at 20:54, Clebert Suconic <[hidden email]
>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726097&i=0>>
wrote:
>>>>>>> 
>>>>>>> Can you cleanup your data (as a test only?) rm -rf data.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> perhaps the spikes you have is the compacting running?
>>>>>>> 
>>>>>>> On Fri, May 12, 2017 at 3:51 PM, Michael André Pearce
>>>>>>> <[hidden email] <http:///user/SendEmail.jtp?
>>> type=node&node=4726097&i=1>>
>>>> 
>>>>>> wrote:
>>>>>>>> All I'm doing is changing Artemis.profile of the instance
to the
>>>>>> different versions.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Sent from my iPhone
>>>>>>>> 
>>>>>>>>> On 12 May 2017, at 20:35, nigro_franz <[hidden email]
>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726097&i=2>>
wrote:
>>>>>>>>> 
>>>>>>>>> Too strange...have you tried with NIO too?It uses always
the same
>>>>>> buffer to
>>>>>>>>> batch writes AFAIK....
>>>>>>>>> 
>>>>>>>>> 2017-05-12 21:30 GMT+02:00 Michael André Pearce [via
ActiveMQ] <
>>>>>>>>> [hidden email] <http:///user/SendEmail.jtp?
>>>> type=node&node=4726097&i=3>>:
>>>>>> 
>>>>>>>>> 
>>>>>>>>>> Ok so it isn't the times buffer changes. Just deployed
2.1.0
>> with
>>>> and
>>>>>>>>>> without the timed buffer changes. Running the same
tests no
>>>>>> difference.
>>>>>>>>>> 
>>>>>>>>>> Must be something else. Causing the behaviour change.
>>>>>>>>>> 
>>>>>>>>>> Sent from my iPhone
>>>>>>>>>> 
>>>>>>>>>>> On 12 May 2017, at 18:10, Michael André Pearce
<[hidden email]
>>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>>
wrote:
>>>>>>>>>>> 
>>>>>>>>>>> I've left for the day now. If I get time over
the weekend I'll
>>> try
>>>>>> see
>>>>>>>>>> if I can make a build of 2.1.0 without that change
and see if it
>>>>>> makes any
>>>>>>>>>> difference.
>>>>>>>>>>> 
>>>>>>>>>>> Can someone point me to the PR for that change,
so I know what
>>> I'm
>>>>>>>>>> unpicking locally?
>>>>>>>>>>> 
>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>> 
>>>>>>>>>>>> On 12 May 2017, at 17:58, Clebert Suconic
<[hidden email]
>>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>>
wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> As the only thing that could affect this
is the Change on
>> timed
>>>>>> buffer.
>>>>>>>>>>>> Afaik
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, May 12, 2017 at 12:57 PM Clebert
Suconic <[hidden
>> email]
>>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> I'm considering only keeping the pooled
buffer part and
>> switch
>>>>>> back to
>>>>>>>>>> the
>>>>>>>>>>>>> ole sleep or an improved sleep we had.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Fri, May 12, 2017 at 12:49 PM Michael
André Pearce <
>>>>>>>>>>>>> [hidden email] <http:///user/SendEmail.jtp?
>>>>>> type=node&node=4726091&i=3>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> As it seems  I can't send images
to mail list, just sent to
>>>> you
>>>>>> both
>>>>>>>>>> via
>>>>>>>>>>>>>> email. Some graphs we have comparing
versions.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Not sure what changes might cause
it.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On 12 May 2017, at 17:37, Clebert
Suconic <[hidden email]
>>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> There is a class we use on producer.
 TokenLimiter.
>> Perhaps
>>>> you
>>>>>>>>>> could
>>>>>>>>>>>>>>> reuse that one ?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Fri, May 12, 2017 at 11:00
AM nigro_franz <[hidden
>> email]
>>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I was thinking of a similar
solution but I've discovered
>>> that
>>>>>>>>>> couldn't
>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>> (in the old or the new TimedBuffer
too), because of the
>>>>>>>>>>>>>>>> TimedBuffer::checkSize method
that could force a flush if
>>> the
>>>>>> batch
>>>>>>>>>>>>>> buffer
>>>>>>>>>>>>>>>> if not big enough to receive
new data, going IOPS.
>>>>>>>>>>>>>>>> Sadly TimedBuffer::checkSize
is outside any timeout, but
>>>>>> depends on
>>>>>>>>>> the
>>>>>>>>>>>>>>>> writers.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> That's why I've implemented
the "compensation" right after
>>>> any
>>>>>>>>>> flush,
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> order to work with forced
flushes too:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/
>>>>>>>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
>>>>>>>>>> src/main/java/org/apache/activemq/artemis/core/io/
>>>>>>>>>> buffer/TimedBuffer.java#L338
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Doing it on TimedBuffer::flush
all the flushes on disk
>> will
>>>> be
>>>>>>>>>>>>>> compensated
>>>>>>>>>>>>>>>> (half of the story: ASYNCIO
is async so depends on libAIO
>>>>>>>>>> partially!)
>>>>>>>>>>>>>>>> Regarding the IOPS computation
I've built this, as you've
>>>>>>>>>> suggested:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/
>>>>>>>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
>>>>>>>>>> src/main/java/org/apache/activemq/artemis/core/io/
>>>>>>>>>> buffer/TimedBuffer.java#L119
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> The performance seems pretty
good, it compensates well but
>>> it
>>>>>> is
>>>>>>>>>> faster
>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>> the original version, limiting
IOPS too!
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Franz
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>>>>>>>>>> IOPS-Limiter-strategy-tp4725875p4726057.html
>>>>>>>>>>>>>>>> Sent from the ActiveMQ -
Dev mailing list archive at
>>>>>> Nabble.com.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Clebert Suconic
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Clebert Suconic
>>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>>> Clebert Suconic
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> ------------------------------
>>>>>>>>>> If you reply to this email, your message will be
added to the
>>>>>> discussion
>>>>>>>>>> below:
>>>>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>>>>>>>>>> IOPS-Limiter-strategy-tp4725875p4726091.html
>>>>>>>>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter
strategy,
>> click
>>>>>> here
>>>>>>>>>> <
>>>>>> 
>>>>>>>>>> .
>>>>>>>>>> NAML
>>>>>>>>>> <http://activemq.2283324.n4.nabble.com/template/
>>>>>> NamlServlet.jtp?macro=macro_viewer&id=instant_html%
>>>>>> 21nabble%3Aemail.naml&base=nabble.naml.namespaces.
>>>>>> BasicNamespace-nabble.view.web.template.NabbleNamespace-
>>>>>> nabble.view.web.template.NodeNamespace&breadcrumbs=
>>>>>> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
>>>>>> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> View this message in context: http://activemq.2283324.n4.
>>>>>> nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-
>>> tp4725875p4726093.html
>>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive at
Nabble.com.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Clebert Suconic
>>>>>> 
>>>>>> 
>>>>>> ------------------------------
>>>>>> If you reply to this email, your message will be added to the
>>>> discussion
>>>>>> below:
>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>>>>>> IOPS-Limiter-strategy-tp4725875p4726097.html
>>>>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click
>>> here
>>>>>> <
>>>>>> .
>>>>>> NAML
>>>>>> <http://activemq.2283324.n4.nabble.com/template/
>>>> NamlServlet.jtp?macro=macro_viewer&id=instant_html%
>>>> 21nabble%3Aemail.naml&base=nabble.naml.namespaces.
>>>> BasicNamespace-nabble.view.web.template.NabbleNamespace-
>>>> nabble.view.web.template.NodeNamespace&breadcrumbs=
>>>> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
>>>> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> View this message in context: http://activemq.2283324.n4.
>>>> 
>> nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726099.html
>>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>>> 
>>>> 
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the
>> discussion
>>>> below:
>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>>>> IOPS-Limiter-strategy-tp4725875p4726101.html
>>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here
>>>> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=
>>> unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0Nz
>>> I1ODc1fC0zNzI3NjE2NQ==>
>>>> .
>>>> NAML
>>>> <http://activemq.2283324.n4.nabble.com/template/
>>> NamlServlet.jtp?macro=macro_viewer&id=instant_html%
>>> 21nabble%3Aemail.naml&base=nabble.naml.namespaces.
>>> BasicNamespace-nabble.view.web.template.NabbleNamespace-
>>> nabble.view.web.template.NodeNamespace&breadcrumbs=
>>> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
>>> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> View this message in context: http://activemq.2283324.n4.
>>> nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726102.html
>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>> 
>> 
> -- 
> Clebert Suconic

Mime
View raw message