Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
MIME-Version: 1.0
In-Reply-To: <6B94E5D2-3428-46FB-AABD-7A543A9DFC34@me.com>
References: <1494405051509-4725875.post@n4.nabble.com> <CAKF+bsoB=YjY-W94pspG48WeCoUUz3WWCh4wGs5GYwZ_pGAKfA@mail.gmail.com>
 <CAKF+bsp285hEXZ8W7WB2mJDkcA2yz9eky7JN0Qy8g67uGDoL1Q@mail.gmail.com>
 <1494446886827-4725924.post@n4.nabble.com> <CAKF+bspm2QVE0sKs5EOx4t9cEPTr1tkm7pC71kk5EUnQwZMe6g@mail.gmail.com>
 <1494447940562-4725927.post@n4.nabble.com> <1494596187032-4726037.post@n4.nabble.com>
 <CAKF+bspfynqz-PCcCG47fmpmsooaijPgC1199myOowWOmQHvTw@mail.gmail.com>
 <CAKF+bsqNzVGBxOSUPXqgMjenGfwyzUMgSUBmbAXx4ke5vxKNOg@mail.gmail.com>
 <1494600482076-4726057.post@n4.nabble.com> <CAKF+bsot2XTCJMAYL2x2pmXkPnrNuGeRjCUye60kWLiOQo6jHg@mail.gmail.com>
 <FD2AB8A0-BE58-4752-9B22-8A9E55577EBB@me.com> <CAKF+bspTTm5=rLT=mrPzGjBkLCmsM8h-sNjwtvxk6JMeDfkPyg@mail.gmail.com>
 <CAKF+bsrezi1ONCHR6cUGkS9ErxtNJmHj0rzZmZ+vYV7wURWtPw@mail.gmail.com>
 <33D5C16C-3C6F-4DDB-85DD-430B5C46EB87@me.com> <6B94E5D2-3428-46FB-AABD-7A543A9DFC34@me.com>
From: Clebert Suconic <clebert.suconic@gmail.com>
Date: Fri, 12 May 2017 15:52:59 -0400
Message-ID: <CAKF+bsqhfMbR_3f182Jrkd2m9n=f+KJ+LF81GnO=_U6_phZ5GA@mail.gmail.com>
Subject: Re: [DISCUSS] Artemis IOPS Limiter strategy
To: "dev@activemq.apache.org" <dev@activemq.apache.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
archived-at: Fri, 12 May 2017 19:53:09 -0000

The only difference in IO applied was:

https://issues.apache.org/jira/browse/ARTEMIS-1117


There's also the MaxSize that we now set the maxSize at half the -Xmx
(if you create a new broker).. that may have some implications on your
GC depending on the test you are doing.. perhaps you could use a
global max defined?

On Fri, May 12, 2017 at 3:42 PM, Michael Andr=C3=A9 Pearce
<michael.andre.pearce@me.com> wrote:
> Ok so it isn't the times buffer changes. Just deployed 2.1.0 with and wit=
hout the timed buffer changes. Running the same tests no difference.
>
> Must be something else. Causing the behaviour change.
>
> Sent from my iPhone
>
>> On 12 May 2017, at 18:10, Michael Andr=C3=A9 Pearce <michael.andre.pearc=
e@me.com> wrote:
>>
>> I've left for the day now. If I get time over the weekend I'll try see i=
f I can make a build of 2.1.0 without that change and see if it makes any d=
ifference.
>>
>> Can someone point me to the PR for that change, so I know what I'm unpic=
king locally?
>>
>> Sent from my iPhone
>>
>>> On 12 May 2017, at 17:58, Clebert Suconic <clebert.suconic@gmail.com> w=
rote:
>>>
>>> As the only thing that could affect this is the Change on timed buffer.
>>> Afaik
>>>
>>>
>>> On Fri, May 12, 2017 at 12:57 PM Clebert Suconic <clebert.suconic@gmail=
.com>
>>> wrote:
>>>
>>>> I'm considering only keeping the pooled buffer part and switch back to=
 the
>>>> ole sleep or an improved sleep we had.
>>>>
>>>>
>>>>
>>>> On Fri, May 12, 2017 at 12:49 PM Michael Andr=C3=A9 Pearce <
>>>> michael.andre.pearce@me.com> wrote:
>>>>
>>>>> As it seems  I can't send images to mail list, just sent to  you both=
 via
>>>>> email. Some graphs we have comparing versions.
>>>>>
>>>>> Not sure what changes might cause it.
>>>>>
>>>>>> On 12 May 2017, at 17:37, Clebert Suconic <clebert.suconic@gmail.com=
>
>>>>> wrote:
>>>>>>
>>>>>> There is a class we use on producer.  TokenLimiter.  Perhaps you cou=
ld
>>>>>> reuse that one ?
>>>>>>
>>>>>>
>>>>>>> On Fri, May 12, 2017 at 11:00 AM nigro_franz <nigro.fra@gmail.com>
>>>>> wrote:
>>>>>>>
>>>>>>> I was thinking of a similar solution but I've discovered that could=
n't
>>>>> work
>>>>>>> (in the old or the new TimedBuffer too), because of the
>>>>>>> TimedBuffer::checkSize method that could force a flush if the batch
>>>>> buffer
>>>>>>> if not big enough to receive new data, going IOPS.
>>>>>>> Sadly TimedBuffer::checkSize is outside any timeout, but depends on=
 the
>>>>>>> writers.
>>>>>>>
>>>>>>> That's why I've implemented the "compensation" right after any flus=
h,
>>>>> in
>>>>>>> order to work with forced flushes too:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>> https://github.com/franz1981/activemq-artemis/blob/4b831021dab3e0dd27=
6f477e3ea665e11ab54d0e/artemis-journal/src/main/java/org/apache/activemq/ar=
temis/core/io/buffer/TimedBuffer.java#L338
>>>>>>>
>>>>>>> Doing it on TimedBuffer::flush all the flushes on disk will be
>>>>> compensated
>>>>>>> (half of the story: ASYNCIO is async so depends on libAIO partially=
!)
>>>>>>> Regarding the IOPS computation I've built this, as you've suggested=
:
>>>>>>>
>>>>>>>
>>>>> https://github.com/franz1981/activemq-artemis/blob/4b831021dab3e0dd27=
6f477e3ea665e11ab54d0e/artemis-journal/src/main/java/org/apache/activemq/ar=
temis/core/io/buffer/TimedBuffer.java#L119
>>>>>>>
>>>>>>> The performance seems pretty good, it compensates well but it is fa=
ster
>>>>>>> than
>>>>>>> the original version, limiting IOPS too!
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Franz
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>>
>>>>> http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-st=
rategy-tp4725875p4726057.html
>>>>>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>>>>>>>
>>>>>> --
>>>>>> Clebert Suconic
>>>>>
>>>> --
>>>> Clebert Suconic
>>>>
>>> --
>>> Clebert Suconic


--=20
Clebert Suconic