jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe Mouawad <philippe.moua...@gmail.com>
Subject Re: Failed samples and aggregate report
Date Fri, 06 Feb 2015 19:20:47 GMT
Hi Vladimir,

On Fri, Feb 6, 2015 at 1:38 PM, Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:

> Hi,
>
> I believe JMeter support of tracking failed samples can be
> significantly improved.
> Each and every manual suggests using "Aggregate Report" ([1]) or
> friends, however it silently returns wrong data without any warning.
>
> For instance: (sample, response time, status)
> Sample1, 9 sec, OK
> Sample1, 0 sec, ERR
> Sample1, 0 sec, ERR
>
> JMeter would show "average response time" as 3 seconds, and same for
> percentiles (median should be 9 seconds, not 0).
> However, it is common for failed requests to run much faster.
> It does not matter very much how fast you can crash, but it does
> matter how fast the successful responses are.
>
> I see two problems here:
> 1) Default configuration averages/computes quantiles for both OK and
> ERR responses. So users get wrong values in the report.
>

Can you open a bugzilla, should be easy to fix

> 2) There is no easy way to track OK and ERR separately. Well, one can
> add _two_ copies of AggregateReport (one for success, another one for
> fails), however is that ever suggested in the documentation? Is that a
> good user experience? One would have to switch back and forth from one
> to another.
>
> Any ideas/plans to fix that?
>
Have a look at that and give us your feedback. Your code contributions are
also very welcome :
- https://issues.apache.org/bugzilla/show_bug.cgi?id=57516


> Well, I do not suggest implementing full set of reporting yet, but it
> is a pity to see how performance engineers report wrong results unless
> you pinpoint all those JMeter pitfalls.
>
> [1]:
> http://jmeter.apache.org/usermanual/component_reference.html#Aggregate_Report
>
>
Not sure this can only be fixed by JMeter :-)

> --
> Regards,
> Vladimir Sitnikov
>



-- 
Cordialement.
Philippe Mouawad.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message