jmeter-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sebb <seb...@gmail.com>
Subject Re: Coordinated Omission (CO) - possible strategies
Date Fri, 18 Oct 2013 11:34:01 GMT
On 18 October 2013 07:47, Kirk Pepperdine <kirk.pepperdine@gmail.com> wrote:
> Hi Sebb,
>
> In my testing, the option off creating threads on demand instead of all at once has made
a huge difference in my being able to control rate of arrivals on the server. It has convinced
me that simply using the throughput controller isn't enough and that the threading model in
JMeter *must* change. It is the threading model that is the biggest source of CO in JMeter.
Unfortunately we weren't able to come to some way of a non-disruptive change in JMeter to
make this happen.
>
> The model I was proposing would have JMeter generate an event heap sorted by the time
when a sampler should be fired. A thread pool should be used to eat off of the heap and fire
the events as per scheduled. This would allow JMeter to break the inappropriate relationship
of a thread being a user. The solution is not perfect in that you will still have to fight
with thread schedulers and hypervisors to get things to happen on queue. However, I believe
the end result will be a far more scalable product that will require far fewer threads to
produce far higher loads on the server.
>
> As for your idea on the using the throughput controller. IHMO triggering an assert only
worsens the CO problem. In fact, if the response times from the timeouts are not added into
the results, in other words they are omitted from the data set, you've only made the problem
worse as you are filter out bad data points from the result sets making the results better
than they should be. Peter Lawyer's (included here for the purpose of this discussion) technique
for correcting CO is to simply recognize when the event should have been triggered and then
start the timer for that event at that time. So the latency reported will include the time
before event triggering.
>
> Gil Tene's done some work with JMeter. I'll leave it up to him to post what he's done.
The interesting bit that he's created is HrdHistogram (https://github.com/giltene/HdrHistogram).
It is not only a better way to report results,it offers techniques to calculate and correct
for CO. Also Gil might be able to point you to a more recent version of his on CO talk. It
might be nice to have a new sampler that incorporates this work.
>
> On a side note, I've got a Servlet filter that is JMX component that measures a bunch
of stats from the servers POV. It's something that could be contributed as it could be used
to help understand the source of CO.. if not just complement JMeter's view of latency.
>

I think you have missed the point of my posting.

The idea was to detect when CO has happened, and use that information
to change the test setup.
In some cases it may not be possible avoid the CO, but in other cases,
it should be possible to reduce the transaction rate in each thread
such that long sample times don't cause the next sample to be delayed.
And at least the user will have the required information.

So, I'll ask again: is my proposal for *detecting* CO reasonable?
If not, what changes are needed?

Changing JMeter to behave differently is a matter for a separate thread.

> Regards,
> Kirk
>
>
> On 2013-10-18, at 12:27 AM, sebb <sebbaz@gmail.com> wrote:
>
>> It looks to be quite difficult to avoid the issue of Coordination
>> Omission without a major redesign of JMeter.
>>
>> However, it may be a lot easier to detect when the condition has occurred.
>> This would potentially allow the test settings to be changed to reduce
>> or eliminate the occurrences - e.g. by increasing the number of
>> threads or spreading the load across more JMeter instances.
>>
>> The Constant Throughput Controller calculates the desired wait time,
>> and if this is less than zero - i.e. a sample should already have been
>> generated - it could trigger the creation of a failed Assertion
>> showing the time difference.
>>
>> Would this be sufficient to detect all CO occurrences?
>> If not, what other metric needs to be checked?
>>
>> Even if it is not the only possible cause, would it be useful as a
>> starting point?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
>> For additional commands, e-mail: user-help@jmeter.apache.org
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@jmeter.apache.org
For additional commands, e-mail: user-help@jmeter.apache.org


Mime
View raw message