activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clebert Suconic <clebert.suco...@gmail.com>
Subject Re: [DISCUSS] Artemis IOPS Limiter strategy
Date Fri, 12 May 2017 14:43:49 GMT
although.. we currently configure the time an operation can be
performed on disk...

you will need to convert the NanoSeconds back to the IOPS you need..
or change your implementation to cope with what we configure now...



We could endup deprecating the previous configuration and use a new
attribute, and multiple the nanoseconds we currently configure to the
new property.

On Fri, May 12, 2017 at 10:20 AM, Clebert Suconic
<clebert.suconic@gmail.com> wrote:
> You could replace the TimedBuffer.sleep for this implementation of
> yours. it has the same semantic...
>
>
> For the implementation (Mapped) that's not using the TimedBuffer you
> may use this directly.
>
> On Fri, May 12, 2017 at 9:36 AM, nigro_franz <nigro.fra@gmail.com> wrote:
>> I've put the first attempt of relaxed/best effort IOPS limiter that works
>> both for ASYNCIO and NIO too.
>>
>> (The second commit on
>> https://github.com/franz1981/activemq-artemis/tree/iops_limiter)
>>
>> The 2 main differences with the original one embedded in the TimedBuffer
>> are:
>>
>> 1) it performs adapting compensation instead of upfront limitation, using
>> the current IOPS perceived
>> 2) it performs compensation against ALL flush operations like the forced
>> ones due to a filled batch buffer, not handled in the original one
>>
>> In practical terms it doesn't overcompensate when the system is not really
>> under stress, but It doesn't let the disk run out of IOPS due to forced
>> batch buffer flushes.
>>
>> I've added in that experimental branch a real-time HdrHistogram
>> instrumentation to record the distributions of:
>>
>> - flushed batch size in bytes (to understand the effectiveness of the
>> coalescing heuristic)
>> - "perceived" flush latency
>> - compensations done by the limiter
>>
>> After some measurements, I've noticed that NIO and ASYNCIO behave very
>> differently.
>> ASYNCIO with maxIO>1 performs async flushes (histograms can't lie), while
>> NIO si always synchronous and blocking, making the compensations more
>> effective.
>> Further investigations could will the best strategy to do it.
>>
>> If you have any idea, curiosity and/or advices about it please share it :)
>>
>> Franz
>>
>>
>>
>>
>> --
>> View this message in context: http://activemq.2283324.n4.nabble.com/DISCUSS-Artemis-IOPS-Limiter-strategy-tp4725875p4726037.html
>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>
>
>
> --
> Clebert Suconic



-- 
Clebert Suconic

Mime
View raw message