activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clebert Suconic <clebert.suco...@gmail.com>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Fri, 12 May 2017 20:33:20 GMT
The version change has little to do with it.. especially between 2.0 and 2.1..

You would probably have the same issue with restarting the broker at
the same version?




On Fri, May 12, 2017 at 4:31 PM, Michael André Pearce
<michael.andre.pearce@me.com> wrote:
> Yup deleting the data on version change removes the issue.
>
> Sent from my iPhone
>
>> On 12 May 2017, at 20:54, Clebert Suconic <clebert.suconic@gmail.com> wrote:
>>
>> Can you cleanup your data (as a test only?) rm -rf data.
>>
>>
>>
>>
>> perhaps the spikes you have is the compacting running?
>>
>> On Fri, May 12, 2017 at 3:51 PM, Michael André Pearce
>> <michael.andre.pearce@me.com> wrote:
>>> All I'm doing is changing Artemis.profile of the instance to the different versions.
>>>
>>>
>>> Sent from my iPhone
>>>
>>>> On 12 May 2017, at 20:35, nigro_franz <nigro.fra@gmail.com> wrote:
>>>>
>>>> Too strange...have you tried with NIO too?It uses always the same buffer
to
>>>> batch writes AFAIK....
>>>>
>>>> 2017-05-12 21:30 GMT+02:00 Michael André Pearce [via ActiveMQ] <
>>>> ml+s2283324n4726091h96@n4.nabble.com>:
>>>>
>>>>> Ok so it isn't the times buffer changes. Just deployed 2.1.0 with and
>>>>> without the timed buffer changes. Running the same tests no difference.
>>>>>
>>>>> Must be something else. Causing the behaviour change.
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>>> On 12 May 2017, at 18:10, Michael André Pearce <[hidden email]
>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>>
wrote:
>>>>>>
>>>>>> I've left for the day now. If I get time over the weekend I'll try
see
>>>>> if I can make a build of 2.1.0 without that change and see if it makes
any
>>>>> difference.
>>>>>>
>>>>>> Can someone point me to the PR for that change, so I know what I'm
>>>>> unpicking locally?
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>>> On 12 May 2017, at 17:58, Clebert Suconic <[hidden email]
>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>>
wrote:
>>>>>>>
>>>>>>> As the only thing that could affect this is the Change on timed
buffer.
>>>>>>> Afaik
>>>>>>>
>>>>>>>
>>>>>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <[hidden
email]
>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I'm considering only keeping the pooled buffer part and switch
back to
>>>>> the
>>>>>>>> ole sleep or an improved sleep we had.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce <
>>>>>>>> [hidden email] <http:///user/SendEmail.jtp?type=node&node=4726091&i=3>>
>>>>> wrote:
>>>>>>>>
>>>>>>>>> As it seems  I can't send images to mail list, just sent
to  you both
>>>>> via
>>>>>>>>> email. Some graphs we have comparing versions.
>>>>>>>>>
>>>>>>>>> Not sure what changes might cause it.
>>>>>>>>>
>>>>>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <[hidden
email]
>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> There is a class we use on producer.  TokenLimiter.
 Perhaps you
>>>>> could
>>>>>>>>>> reuse that one ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz
<[hidden email]
>>>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> I was thinking of a similar solution but I've
discovered that
>>>>> couldn't
>>>>>>>>> work
>>>>>>>>>>> (in the old or the new TimedBuffer too), because
of the
>>>>>>>>>>> TimedBuffer::checkSize method that could force
a flush if the batch
>>>>>>>>> buffer
>>>>>>>>>>> if not big enough to receive new data, going
IOPS.
>>>>>>>>>>> Sadly TimedBuffer::checkSize is outside any timeout,
but depends on
>>>>> the
>>>>>>>>>>> writers.
>>>>>>>>>>>
>>>>>>>>>>> That's why I've implemented the "compensation"
right after any
>>>>> flush,
>>>>>>>>> in
>>>>>>>>>>> order to work with forced flushes too:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/
>>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
>>>>> src/main/java/org/apache/activemq/artemis/core/io/
>>>>> buffer/TimedBuffer.java#L338
>>>>>>>>>>>
>>>>>>>>>>> Doing it on TimedBuffer::flush all the flushes
on disk will be
>>>>>>>>> compensated
>>>>>>>>>>> (half of the story: ASYNCIO is async so depends
on libAIO
>>>>> partially!)
>>>>>>>>>>> Regarding the IOPS computation I've built this,
as you've
>>>>> suggested:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>> https://github.com/franz1981/activemq-artemis/blob/
>>>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
>>>>> src/main/java/org/apache/activemq/artemis/core/io/
>>>>> buffer/TimedBuffer.java#L119
>>>>>>>>>>>
>>>>>>>>>>> The performance seems pretty good, it compensates
well but it is
>>>>> faster
>>>>>>>>>>> than
>>>>>>>>>>> the original version, limiting IOPS too!
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Franz
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context:
>>>>>>>>>>>
>>>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>>>>> IOPS-Limiter-strategy-tp4725875p4726057.html
>>>>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive
at Nabble.com.
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Clebert Suconic
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Clebert Suconic
>>>>>>>>
>>>>>>> --
>>>>>>> Clebert Suconic
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> If you reply to this email, your message will be added to the discussion
>>>>> below:
>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
>>>>> IOPS-Limiter-strategy-tp4725875p4726091.html
>>>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here
>>>>> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0NzI1ODc1fC0zNzI3NjE2NQ==>
>>>>> .
>>>>> NAML
>>>>> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726093.html
>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>
>>
>>
>> --
>> Clebert Suconic



-- 
Clebert Suconic

Mime
View raw message