activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nigro_franz <nigro....@gmail.com>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Fri, 12 May 2017 20:24:27 GMT
Hence the new TimedBuffer didn't change drastically the latencies?
I've got some graph after the stress test with a maxed IO, if you're
curious :)

2017-05-12 22:19 GMT+02:00 Michael André Pearce [via ActiveMQ] <
ml+s2283324n4726097h82@n4.nabble.com>:

> Yup deleting the data on version change removes the issue.
>
> Sent from my iPhone
>
> > On 12 May 2017, at 20:54, Clebert Suconic <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4726097&i=0>> wrote:
> >
> > Can you cleanup your data (as a test only?) rm -rf data.
> >
> >
> >
> >
> > perhaps the spikes you have is the compacting running?
> >
> > On Fri, May 12, 2017 at 3:51 PM, Michael André Pearce
> > <[hidden email] <http:///user/SendEmail.jtp?type=node&node=4726097&i=1>>
> wrote:
> >> All I'm doing is changing Artemis.profile of the instance to the
> different versions.
> >>
> >>
> >> Sent from my iPhone
> >>
> >>> On 12 May 2017, at 20:35, nigro_franz <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4726097&i=2>> wrote:
> >>>
> >>> Too strange...have you tried with NIO too?It uses always the same
> buffer to
> >>> batch writes AFAIK....
> >>>
> >>> 2017-05-12 21:30 GMT+02:00 Michael André Pearce [via ActiveMQ] <
> >>> [hidden email] <http:///user/SendEmail.jtp?type=node&node=4726097&i=3>>:
>
> >>>
> >>>> Ok so it isn't the times buffer changes. Just deployed 2.1.0 with and
> >>>> without the timed buffer changes. Running the same tests no
> difference.
> >>>>
> >>>> Must be something else. Causing the behaviour change.
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>>> On 12 May 2017, at 18:10, Michael André Pearce <[hidden email]
> >>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>>
wrote:
> >>>>>
> >>>>> I've left for the day now. If I get time over the weekend I'll try
> see
> >>>> if I can make a build of 2.1.0 without that change and see if it
> makes any
> >>>> difference.
> >>>>>
> >>>>> Can someone point me to the PR for that change, so I know what I'm
> >>>> unpicking locally?
> >>>>>
> >>>>> Sent from my iPhone
> >>>>>
> >>>>>> On 12 May 2017, at 17:58, Clebert Suconic <[hidden email]
> >>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>>
wrote:
> >>>>>>
> >>>>>> As the only thing that could affect this is the Change on timed
> buffer.
> >>>>>> Afaik
> >>>>>>
> >>>>>>
> >>>>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <[hidden
email]
> >>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I'm considering only keeping the pooled buffer part and
switch
> back to
> >>>> the
> >>>>>>> ole sleep or an improved sleep we had.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce <
> >>>>>>> [hidden email] <http:///user/SendEmail.jtp?
> type=node&node=4726091&i=3>>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> As it seems  I can't send images to mail list, just
sent to  you
> both
> >>>> via
> >>>>>>>> email. Some graphs we have comparing versions.
> >>>>>>>>
> >>>>>>>> Not sure what changes might cause it.
> >>>>>>>>
> >>>>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <[hidden
email]
> >>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> There is a class we use on producer.  TokenLimiter.
 Perhaps you
> >>>> could
> >>>>>>>>> reuse that one ?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz
<[hidden email]
> >>>> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I was thinking of a similar solution but I've
discovered that
> >>>> couldn't
> >>>>>>>> work
> >>>>>>>>>> (in the old or the new TimedBuffer too), because
of the
> >>>>>>>>>> TimedBuffer::checkSize method that could force
a flush if the
> batch
> >>>>>>>> buffer
> >>>>>>>>>> if not big enough to receive new data, going
IOPS.
> >>>>>>>>>> Sadly TimedBuffer::checkSize is outside any
timeout, but
> depends on
> >>>> the
> >>>>>>>>>> writers.
> >>>>>>>>>>
> >>>>>>>>>> That's why I've implemented the "compensation"
right after any
> >>>> flush,
> >>>>>>>> in
> >>>>>>>>>> order to work with forced flushes too:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>> https://github.com/franz1981/activemq-artemis/blob/
> >>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
> >>>> src/main/java/org/apache/activemq/artemis/core/io/
> >>>> buffer/TimedBuffer.java#L338
> >>>>>>>>>>
> >>>>>>>>>> Doing it on TimedBuffer::flush all the flushes
on disk will be
> >>>>>>>> compensated
> >>>>>>>>>> (half of the story: ASYNCIO is async so depends
on libAIO
> >>>> partially!)
> >>>>>>>>>> Regarding the IOPS computation I've built this,
as you've
> >>>> suggested:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>> https://github.com/franz1981/activemq-artemis/blob/
> >>>> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
> >>>> src/main/java/org/apache/activemq/artemis/core/io/
> >>>> buffer/TimedBuffer.java#L119
> >>>>>>>>>>
> >>>>>>>>>> The performance seems pretty good, it compensates
well but it
> is
> >>>> faster
> >>>>>>>>>> than
> >>>>>>>>>> the original version, limiting IOPS too!
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Franz
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> View this message in context:
> >>>>>>>>>>
> >>>>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
> >>>> IOPS-Limiter-strategy-tp4725875p4726057.html
> >>>>>>>>>> Sent from the ActiveMQ - Dev mailing list archive
at
> Nabble.com.
> >>>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Clebert Suconic
> >>>>>>>>
> >>>>>>> --
> >>>>>>> Clebert Suconic
> >>>>>>>
> >>>>>> --
> >>>>>> Clebert Suconic
> >>>>
> >>>>
> >>>> ------------------------------
> >>>> If you reply to this email, your message will be added to the
> discussion
> >>>> below:
> >>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
> >>>> IOPS-Limiter-strategy-tp4725875p4726091.html
> >>>> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click
> here
> >>>> <
>
> >>>> .
> >>>> NAML
> >>>> <http://activemq.2283324.n4.nabble.com/template/
> NamlServlet.jtp?macro=macro_viewer&id=instant_html%
> 21nabble%3Aemail.naml&base=nabble.naml.namespaces.
> BasicNamespace-nabble.view.web.template.NabbleNamespace-
> nabble.view.web.template.NodeNamespace&breadcrumbs=
> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> View this message in context: http://activemq.2283324.n4.
> nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726093.html
> >>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
> >
> >
> >
> > --
> > Clebert Suconic
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
> IOPS-Limiter-strategy-tp4725875p4726097.html
> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0NzI1ODc1fC0zNzI3NjE2NQ==>
> .
> NAML
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726099.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message