jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe Mouawad <philippe.moua...@gmail.com>
Subject Re: Throughput timers vs Poisson vs etc
Date Fri, 06 Feb 2015 21:38:28 GMT
On Fri, Feb 6, 2015 at 9:32 PM, Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:

> > Did you try Constant Throughput Timer
>
> I did, however it does have drawbacks.
> 1.1) It does not always work. Sorry, it was long ago I've last tested
> it, so I cannot list lots of examples from the top of my head.
> For instance, I've just created 100 threads, set thread duration to be
> 20 seconds and put constant throughput timer of 60 per minute.
> I expect exactly 20 items of "debug sampler" to appear, but 100 happens.
> Test plan:
> ThreadGroup(100 threads, 20 seconds test duration)
>   Loop(Forever)
>     DebugSampler(for_throughput)
>       ConstantThroughputTimer(60 per minute)
>     DebugSampler(work)
>

I suggest you open bugs with if possible jmx plan showing issue.


>
> 1.2) It tries to adapt to the target based on the duration of the loop
> execution.
> It works really awful for long iterations. For instance, if a single
> iteration takes 10 minutes and I want 10 per hour, then CTT has just
> no time to "adapt".
> If an iteration takes longer than CTT expected, then you're screwed.
> You get less executions than required.
>
> 1.3) CTT is not reproducible. See point 3 of original email on
> "Repeatable test profile".
>
> 1.4) CTT tries to do even delay between iterations. This plays awful
> when you have multiple scenarios running at once if the scenarios have
> the same target throughput. You end up with sin(t) kind of CPU%.
>
> 5) "Bursty load without explicit batches". If I set CTT at 1 per
> minute, it will never launch iterations closer than 60 seconds
> (assuming all the bugs like 1.1 are fixed). However, in reality
> arrivals often have spikes. For instance, in poisson process (those
> kind of arrivals are common to happen in telecom industry)
> inter-arrival durations vary (bingo!).
> In other words, if an action takes 30 seconds to execute and CTT
> launches it every 60 seconds, then this kind of testing will NEVER
> catch a case when two actions run concurrently. With inter-arrival
> delays like 10 sec, 110 sec, 80 sec, 40 sec throughput would still
> result in "1 per minute" on average, but it will catch more
> performance bugs (e.g. the one when two concurrent actions result in
> out of memory).
>
> > Yes you can, just schedule them :
> > Scheduler > Startup Delay + Duration
>
> It does not scale: suppose you have 20 thread groups (e.g. 20 UI
> scenarios).
> How would you manage all those durations?
> How would you change the duration if throughput needs to be changed?
> (hint: in scalability tests throughput is often need to be altered for
> each run)
> How would you pass the script to your buddy?
>

Okay.
Did you try this one from 3rd party:
http://jmeter-plugins.org/wiki/ThroughputShapingTimer/

>
> Even if you could configure it once, those 0:0:0 defaults will
> definitely bite you very soon (as a new group is added).
>
> Vladimir
>

Thanks for this feedback,  it will help us improve.
Regards

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message