activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nigro_franz <nigro....@gmail.com>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Fri, 12 May 2017 19:35:39 GMT
Too strange...have you tried with NIO too?It uses always the same buffer to
batch writes AFAIK....

2017-05-12 21:30 GMT+02:00 Michael André Pearce [via ActiveMQ] <
ml+s2283324n4726091h96@n4.nabble.com>:

> Ok so it isn't the times buffer changes. Just deployed 2.1.0 with and
> without the timed buffer changes. Running the same tests no difference.
>
> Must be something else. Causing the behaviour change.
>
> Sent from my iPhone
>
> > On 12 May 2017, at 18:10, Michael André Pearce <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4726091&i=0>> wrote:
> >
> > I've left for the day now. If I get time over the weekend I'll try see
> if I can make a build of 2.1.0 without that change and see if it makes any
> difference.
> >
> > Can someone point me to the PR for that change, so I know what I'm
> unpicking locally?
> >
> > Sent from my iPhone
> >
> >> On 12 May 2017, at 17:58, Clebert Suconic <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4726091&i=1>> wrote:
> >>
> >> As the only thing that could affect this is the Change on timed buffer.
> >> Afaik
> >>
> >>
> >> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4726091&i=2>>
> >> wrote:
> >>
> >>> I'm considering only keeping the pooled buffer part and switch back to
> the
> >>> ole sleep or an improved sleep we had.
> >>>
> >>>
> >>>
> >>> On Fri, May 12, 2017 at 12:49 PM Michael André Pearce <
> >>> [hidden email] <http:///user/SendEmail.jtp?type=node&node=4726091&i=3>>
> wrote:
> >>>
> >>>> As it seems  I can't send images to mail list, just sent to  you both
> via
> >>>> email. Some graphs we have comparing versions.
> >>>>
> >>>> Not sure what changes might cause it.
> >>>>
> >>>>> On 12 May 2017, at 17:37, Clebert Suconic <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4726091&i=4>>
> >>>> wrote:
> >>>>>
> >>>>> There is a class we use on producer.  TokenLimiter.  Perhaps you
> could
> >>>>> reuse that one ?
> >>>>>
> >>>>>
> >>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4726091&i=5>>
> >>>> wrote:
> >>>>>>
> >>>>>> I was thinking of a similar solution but I've discovered that
> couldn't
> >>>> work
> >>>>>> (in the old or the new TimedBuffer too), because of the
> >>>>>> TimedBuffer::checkSize method that could force a flush if the
batch
> >>>> buffer
> >>>>>> if not big enough to receive new data, going IOPS.
> >>>>>> Sadly TimedBuffer::checkSize is outside any timeout, but depends
on
> the
> >>>>>> writers.
> >>>>>>
> >>>>>> That's why I've implemented the "compensation" right after any
> flush,
> >>>> in
> >>>>>> order to work with forced flushes too:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>> https://github.com/franz1981/activemq-artemis/blob/
> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
> src/main/java/org/apache/activemq/artemis/core/io/
> buffer/TimedBuffer.java#L338
> >>>>>>
> >>>>>> Doing it on TimedBuffer::flush all the flushes on disk will
be
> >>>> compensated
> >>>>>> (half of the story: ASYNCIO is async so depends on libAIO
> partially!)
> >>>>>> Regarding the IOPS computation I've built this, as you've
> suggested:
> >>>>>>
> >>>>>>
> >>>> https://github.com/franz1981/activemq-artemis/blob/
> 4b831021dab3e0dd276f477e3ea665e11ab54d0e/artemis-journal/
> src/main/java/org/apache/activemq/artemis/core/io/
> buffer/TimedBuffer.java#L119
> >>>>>>
> >>>>>> The performance seems pretty good, it compensates well but it
is
> faster
> >>>>>> than
> >>>>>> the original version, limiting IOPS too!
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Franz
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> View this message in context:
> >>>>>>
> >>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
> IOPS-Limiter-strategy-tp4725875p4726057.html
> >>>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
> >>>>>>
> >>>>> --
> >>>>> Clebert Suconic
> >>>>
> >>> --
> >>> Clebert Suconic
> >>>
> >> --
> >> Clebert Suconic
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-
> IOPS-Limiter-strategy-tp4725875p4726091.html
> To unsubscribe from [DISCUSS] Artemis IOPS Limiter strategy, click here
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4725875&code=bmlncm8uZnJhQGdtYWlsLmNvbXw0NzI1ODc1fC0zNzI3NjE2NQ==>
> .
> NAML
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726093.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message