Mailing-List: contact dev-help@jmeter.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@jmeter.apache.org
Received-SPF: pass (nike.apache.org: domain of philippe.mouawad@gmail.com
 designates 209.85.213.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAH9fUpYV1SJBvWZkKhnoHUX_n7EZBaSAczzDQK3yX8FQDm-SHg@mail.gmail.com>
References: 
 <CAH9fUpYV1SJBvWZkKhnoHUX_n7EZBaSAczzDQK3yX8FQDm-SHg@mail.gmail.com>
Date: Sun, 16 Nov 2014 14:49:47 +0100
Message-ID: 
 <CAH9fUpbmiDTJdifCAuq6NSX10WoyQiaRNxaWZG4LXVcxK8moog@mail.gmail.com>
Subject: Re: Graphs and reporting in Apache JMeter : Today and Future
From: Philippe Mouawad <philippe.mouawad@gmail.com>
To: "dev@jmeter.apache.org" <dev@jmeter.apache.org>
Content-Type: multipart/alternative; boundary=001a113ece509d9cb20507fa225f

--001a113ece509d9cb20507fa225f
Content-Type: text/plain; charset=ISO-8859-1

I created https://issues.apache.org/bugzilla/show_bug.cgi?id=57222 for the
similarities between Aggregate Report and Summary Report

Regards
Philippe M.
@philmdot

On Sun, Nov 16, 2014 at 2:32 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:

> Hello,
> I would like to start a discussion about the future of graphs and
> reporting in Apache JMeter.
>
> Todays state:
>
>    - My feeling (I don't want to hurt anybody who worked on those
>    listeners which are interesting and I am aware that what can be great at
>    some time can become less few years after) is that today:
>       -  we have the following graphs
>       - Aggregate Graph : currently affected by bug on
>          https://issues.apache.org/bugzilla/show_bug.cgi?id=57221 in Java
>          8
>          - Response Time Graph : interesting but slow to generate and I
>          get OOM when I try to reload an existing CSV in some configuration, I will
>          report a bug
>          - Distribution Graph (alpha) : Alpha is not encouraging for
>          users, is Alpha really needed or is it stable
>          - Graph Results : Does not look very nice compared with what you
>          can get today
>          -  we have the following reports:
>          - Aggregate Report : Interesting
>          - Summary Report: Differs from  Aggregate Report by having
>          additional Avg Bytes and Std Dev but not Median nor 90%Line
>
> My opinion and analysis on the current state is the following:
>
>    - We lack of clean reporting that would be generated automatically at
>    end of load test, all the previous graphs need to be generated from GUI and
>    makes you lose some time. Also as we discourage Load Testing from GUI we
>    should propose a way to generate reporting from NON GUI Mode
>    - Some graphs (Response Time Graph, Aggregate Graph) rely on JCharts
>    which seems to be abandoned, last release is from 2004 and current bug
>    57221 may be related to a bug in the library ,by the way this is an
>    additional argument to get rid of obsolete libraries in JMeter:
>       - Avalon (Logging + DataSource)
>       - Excalibur (logging + DataSource)
>       - jCharts
>       - Reports are not sufficient and partly redundant , for example it
>    seems that in terms of information Aggregate report and Summary Report
>    should be merged. I looked at their code they are really very similar , I
>    think one should be removed and the  infos in both merged
>
>
> I must reckon that today my colleagues and I for example , we all use
> JMeter-Plugins graphs combined with GraphGeneratorListener, and we are very
> happy with what we get:
>
>    -
>    http://www.ubik-ingenierie.com/blog/automatically-generating-nice-graphs-at-end-of-your-load-test-with-apache-jmeter-and-jmeter-plugins/
>
> I would be interested to know how users generate their reports and graphs
> currently.
>
>
> But some may object that JMeter-Plugins is not JMeter and conclude that
> JMeter lacks of an essential feature.
>
> So I have 2 propositions to improve JMeter in this field:
>
>    - 1/ Create a Brand new Report Generator Listener that would at end of
>    load test generate a report in HTML or PDF containing:
>       - Merge of informations contained in current reports
>       - Generate graphs based on JS Libraries, I am impressed by the
>       number of libraries that generate nice and sexy graphs:
>          - http://www.chartjs.org/
>          - https://github.com/keen/dashboards/tree/gh-pages/examples
>       - Having them in HTML would allow some dynamice behaviour but the
>       ideal result would be to be able to export graphs also as images
>    - 2/ I don't know if it is possible but what about merging
>    JMeter-Plugins graphing suite + GraphGeneratorListener in JMeter core if no
>    library is in conflict in terms of licensing, of course it would required
>    their agreement :-)
>
>
> I think today any Load Testing tool comes with  reporting feature wether
> Commercial and Open Source and JMeter as such should improve in this field
> to stay one of the leaders in Load Testing field.
>
> Thoughts ?
>
> --
> Regards
> Philippe M.
> @philmdot
>
>


-- 
Cordialement.
Philippe Mouawad.

--001a113ece509d9cb20507fa225f--