jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Brosig <fabro...@gmail.com>
Subject Re: (Exponentially Distributed) CycleTime instead of ThinkTime in JMeter
Date Thu, 11 Jul 2013 17:52:57 GMT
Hi Adrian,

Thank you for your detailed answer!

Basically, when you make some performance testing, there are two options:
1) You would like to simulate a number of users with a certain think time
(also referred to as Closed Workload).
Then the Constant (Delay) Timer is the right choice.
In case of distributed think times, which is more realistic, one can use
the Random Timers.

2) You would like to simulate a certain arrival rate (i.e., reach a certain
throughput level) (also referred to as Open Workload).
Then the Constant Throughput Timer is an appropriate choice. (How many
threads one should configure is a different question, though relevant)
The Constant Throughput Timer works with a constant cycle time. For
example, I configured 10 threads and would like to achieve a throughput of
1 request per second. Then the cycle time for each thread is computed to
constantly 10 seconds. In particular in case of rather constant request
response times, the load shape will be quite boring, i.e., constant :)
A more realistic shape to achieve a certain throughput level would be a
cycle time which is, e.g., exponentially distributed (see, e.g., [1]). This
use case cannot be captured by the available JMeter Timers. I cannot use
the existing Poisson Timer because its induced arrival rate depends on the
response times I do not know beforehand.


>If resp times degrade over time (justifiable by new features added), you
>wouldn't have to adjust test scripts, with the cycle timer option, you'd
>get the same amount of actions that you would have expected when you first
>designed the test.
Yes, with the cycle timer action, I would always get the same throughput
level (of course, only if the system is able to sustain the injected load).
And if the resp times are degrading because of new features, then I can see
the degradation for that certain throughput level.

>But we do have something similar at the moment, I use the Constant
Throughput Timer for this, with the per thread option. Is this not enough?
Then the load variation might not be realistic. In fact, the load variation
then depends on the variation of the request response times.

>From my point of view, the cycle time option would thus be an interesting
feature. However, I have to admit it might confuse new JMeter users.

Thanks,
Fabian

[1] "The Palm–Khintchine
theorem<http://en.wikipedia.org/wiki/Palm%E2%80%93Khintchine_theorem>
provides
a result that shows that the superposition of many low intensity
non-Poisson point processes will be close to a Poisson process."
http://en.wikipedia.org/wiki/Poisson_process



2013/7/10 Adrian Speteanu <asp.adieu@gmail.com>

> Hi,
>
> The concept is simple enough and interesting. But I think you can already
> achieve what you're asking for.
>
> Why would it be needed in the case of random timers? Let's say you have
> cycle timer, and you go for a random period. Isn't the period added by the
> cycletime minus existing thinktime covered in the deviation? If I'd want to
> simulate what a single user does in a cycle, and the response times of the
> app get in my way, I'd just adjust the offset and the deviation, at the
> moment and I obtain random cycle times in the same range as if I could
> achieve if I had the cycle timer option. It would add extra calculation and
> little benefit. I fail to understand why its needed, but would support the
> idea, because a lot of people struggle to understand how JMeter generates
> its throughput, why v-users's actions get out of sync after the first few
> thread loops. With a cycle time, it would be easier to synchronise threads
> if its really needed. But its not a usecase that should be recommendable.
>
> This is how I understand your request (an example of why I think the
> functionality is available in a different form), please correct me if my
> understanding is wrong:
>
> If resp times degrade over time (justifiable by new features added), you
> wouldn't have to adjust test scripts, with the cycle timer option, you'd
> get the same amount of actions that you would have expected when you first
> designed the test. This is one case where I would like such control of a
> single threads throughput. But we do have something similar at the moment,
>  I use the Constant Throughput Timer for this, with the per thread option.
> Is this not enough?
>
> Cheers,
> Adrian Speteanu
>
> On Tue, Jul 9, 2013 at 7:40 PM, Fabian Brosig <fabrosig@gmail.com> wrote:
>
> > Hi,
> >
> > In my JMeter script, I wanted to configure a timer with an exponential
> > distributed delay.
> > However, using the Poisson timer, I could configure only the think time,
> > not the cycle time. (In short: cycleTime = responseTime + thinkTime)
> >
> > Here is a more detailed explanation, what "Cycle Time" is, and why it is
> > useful in load testing scenarios... (citation from
> > http://www.testnscale.com/docs/CycleTimesTutorial.html)
> > "Cycle Time in Faban refers to the inter-arrival time between subsequent
> > requests that arrive at a server. For internet-based applications where
> the
> > user population is unknown, it makes sense to model cycle times (rather
> > than think times).  Note that there is a subtle difference between using
> > think times and cycle times. In a load test that uses think times, if the
> > server slows down and response times increase, the think times are still
> > generated in the same manner giving the server time to recover. But in a
> > test that uses cycle times, if the response time is large, the cycle time
> > still remains the same - the emulated user simply sleeps for a shorter
> time
> > between requests (and in the worst case where the response time is larger
> > than the cycle time, will not sleep at all) causing the load on the
> server
> > to increase. This will quickly result in excessive load on the server
> > causing increasingly larger response times.  This is not necessarily a
> bad
> > thing - it can pin-point scalability problems in the server more quickly
> > and help you fix them."
> >
> > If I want to achieve a certain throughput level, configuring the
> ThinkTime
> > is not useful, since the actual throughput depends on the response
> times. I
> > now implemented an own PoissonTimer that times the cycle time, not the
> > think time.
> > (The implementation extends the standard PoissonTimer, and uses code
> > fragments of the ConstantThroughputTimer.)
> >
> > My question:
> > From my point of view, it would be definitely useful to provide at least
> > all RandomTimers with an option "control think time or cycle time?". The
> > implementation is rather straight-forward, and I really would like to
> help
> > with that. What do you think?
> >
> > Note: Using the constant throughput timer is not an option, because the
> > cycle times are then constant (see implementation of
> > ConstantThroughputTimer), which is not really realistic/representative.
> >
> > Kind Regards,
> > Fabian
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message