jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antonio Gomes Rodrigues <ra0...@gmail.com>
Subject Re: New Listener to generate HTML report at end of load test Was APDEX Computing and reporting
Date Tue, 10 Feb 2015 08:59:50 GMT
Hi,

Same opinion and all the other load testing tools (Gatling, LoadRunner,
Silkperformer, etc.) have it

Antonio

2015-02-10 9:56 GMT+01:00 <Pascal.Schumacher@t-systems.com>:

> As a user I think this feature would be very helpful.
>
> Reports are a JMeter weakest feature, so enhancements in this area are
> very welcome.
>
> It would also be nice if it as easy as possible to use. Imho this means
> that it should be added to JMeter and not to a separate commandline utility
> etc.
>
> Regards,
> Pascal
>
> -----Urspr√ľngliche Nachricht-----
> Von: Felix Schumacher [mailto:felix.schumacher@internetallee.de]
> Gesendet: Montag, 9. Februar 2015 18:55
> An: dev@jmeter.apache.org
> Betreff: Re: New Listener to generate HTML report at end of load test Was
> APDEX Computing and reporting
>
> Am 03.02.2015 um 13:38 schrieb Philippe Mouawad:
> > Hi,
> > Any additional opinions of dev team or users ?
> > I 'd like to know if I should continue efforts on this or stop it.
> I think it would be a nice feature to have a simple report at the end of a
> jmeter run. Maybe the user should ask for it by specifying a command line
> parameter like "--report".
>
> If this is integrated in the shell wrapper it could even be a program that
> is started after the jmeter run. But I don't think users will generally
> start another program themselves, so we shouldn't hide it too deep.
>
> Regards
>   Felix
> >
> > Thanks
> > Regards
> >
> > On Monday, February 2, 2015, Philippe Mouawad
> > <philippe.mouawad@gmail.com>
> > wrote:
> >
> >>
> >> On Monday, February 2, 2015, sebb <sebbaz@gmail.com
> >> <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
> >>
> >>> On 1 February 2015 at 22:41, Philippe Mouawad
> >>> <philippe.mouawad@gmail.com> wrote:
> >>>> On Sunday, February 1, 2015, sebb <sebbaz@gmail.com> wrote:
> >>>>
> >>>>> On 1 February 2015 at 19:31, Philippe Mouawad
> >>>>> <philippe.mouawad@gmail.com <javascript:;>> wrote:
> >>>>>> Hi ,
> >>>>>> I uploaded a screenshot and new patch code showing what I meant.
> >>>>>> I think including it in core should be considered seriously.
> >>>>> If the code reads the CSV file at the end of a test to produce the
> >>>>> HTML file, there is no need for it to be included in the JMeter
> >>>>> code which is used to run tests.
> >>>>> It should be a separate application.
> >>>>>
> >>>>> I don't share your opinion sebb.
> >>>> Having a report ready at end of load test is nowadays a standard,
> >>>> look
> >>> at
> >>>> market tools wether open source or not.
> >>>> I don't see the problem here, can you give more details ?
> >>> The problem is that it adds extra code to the JMeter application
> >>> which takes extra memory.
> >>
> >> My last implementation only uses memory at end of test not morz during
> it.
> >>
> >>
> >>> Also, if the user wants to process existing files, it will be easier
> >> to have a standalone application rather than having to start the
> >>> JMeter test application and then find the reporting tool.
> >>
> >> It is possible with it, just make a plan with 1 debug sampler and put
> >> listener. Report will be generated.
> >>
> >>> It would be useful if the tool could be used in batch mode to
> >>> process a set of CSV files.
> >>
> >> Maybe, but I think it's an additional feature. Also having another
> >> tool may lead to what happened to the report package, it seems it was
> >> never used.  But if you implement it it will be ok for me.
> >>
> >>
> >>> It's also likely to be easier for others to contribute additional
> >>> reporting tools if they are part of a separate application.
> >> I don't understand this ? how is it easier if it's in jmeter core. If
> >> it's apart I don't support this option.
> >>
> >>
> >>
> >>> It should make the coding requirements simpler.
> >> how ?
> >>
> >>>> Having another application( by the way do you mean provided as a
> >>>> tool in jmeter or third party) means in the first case you need to
> >>>> setup another tool and in the latter case need to develop your own,
> or use or pay one.
> >>> Of course the code should be included with JMeter.
> >>> But it should be a separate tool.
> >>>
> >>>> This is the major drawback of JMeter I hear from customers and read
> >>> around
> >>>> the net.
> >>>> I first though it would not be that easy but it appears we are able
> >>>> to reuse existing code.
> >>> I don't follow.
> >>>
> >>>> If you disagree, I think we should wait for other team members
> >>>> their opinion and/or setup a vote for this.
> >>> The main JMeter code should be reserved for setting up and running
> tests.
> >>>
> >> IMHO reporting must be part of JMeter, and anyway it is already
> >> through some Listeners , which although not perfect bring useful
> >> infos. The aim of this new listener is:
> >> 1/ Allow generation in gui and non gui mode 2/ Have a "sexy" (not yet
> >> ) report readable in browser with dynamic behaviour (zoom, select
> >> some samples...) 3/ Allow generation at end of load test 4/ Make it
> >> easily customizable as FTL will be a property 5/ Make it extensible
> >> in the future
> >>
> >>
> >>
> >>
> >>>>
> >>>>
> >>>>
> >>>>>> @Felix thanks for review of initial patch.
> >>>>>> Regards
> >>>>>>
> >>>>>> On Sunday, February 1, 2015, sebb <sebbaz@gmail.com
> >>>>>> <javascript:;>>
> >>>>> wrote:
> >>>>>>> On 31 January 2015 at 13:16, Philippe Mouawad
> >>>>>>> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>>
wrote:
> >>>>>>>> On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
> >>>>>>>> felix.schumacher@internetallee.de <javascript:;>
> >>>>>>>> <javascript:;>>
> >>>>> wrote:
> >>>>>>>>> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
> >>>>>>>>>
> >>>>>>>>>> Hi,
> >>>>>>>>>> I intend to commit a BackendListener client
implementation
> >>>>>>>>>> that
> >>>>>>> computes
> >>>>>>>>>> APDEX at end of Load Test.
> >>>>>>>>>> It will take:
> >>>>>>>>>>
> >>>>>>>>>>      - Acceptable Response Time (http://www.apdex.fr/)
T
> >>>>>>>>>>
> >>>>>>>>> The link was in french (my french is not very good
(really non
> >>>>> existant
> >>>>>>>>> :)), but luckily wikipedia had an article about
apdex, which I
> >>> could
> >>>>>>> read.
> >>>>>>>> Sorry for the french link some references:
> >>>>>>>> http://www.apdex.org/overview.html
> >>>>>>>>
> >>>>>>>>       - Compute F as 4xT but allow customization
> >>>>>>>>>>      - List of samples taken into account
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> This listener will compute it during load test
and generate
> >>>>>>>>>> an
> >>> HTML
> >>>>>>> file
> >>>>>>>>>> at
> >>>>>>>>>> end of test containing it.
> >>>>>>>>>>
> >>>>>>>>>> This listener could be later enhanced to :
> >>>>>>>>>>
> >>>>>>>>>>      - Generate equivalent of Aggregate Report
> >>>>>>>>>>      - Possibly graphs based on chartjs or some
other
> >>>>>>>>>> graphing JS
> >>>>>>> Library
> >>>>>>>>> I understand, that apdex is a simple mean to have
one metric.
> >>>>>>>>> As
> >>>>> such I
> >>>>>>>>> wonder why we would want to graph it.
> >>>>>>>> In fact my proposal was larger than just APDEX.
> >>>>>>>> What I am proposing is to implement a report generator
that at
> >>> end of
> >>>>>>> Test
> >>>>>>>> would compute HTML page with:
> >>>>>>>> - APDEX
> >>>>>>>> - Some graphs like:
> >>>>>>>> - Plot of OK/KO
> >>>>>>>> (http://www.chartjs.org/docs/#doughnut-pie-chart
> >>> )
> >>>>>>>> - Request Summary (what's currently in Aggregate Graph)
> >>>>>>>> http://www.chartjs.org/docs/#bar-chart
> >>>>>>>> - Response Time over time (
> >>> http://www.chartjs.org/docs/#line-chart)
> >>>>>>>> I think all this should in fact be computed at end of
load test
> >>>>> instead
> >>>>>>> of
> >>>>>>>> during it.
> >>>>>>>> The input of this listener would be the generated CSV
file, and
> >>>>>>>> at
> >>>>> end of
> >>>>>>>> Test it would read it an compute all this.
> >>>>>>> In which case it should be a stand-alone app.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> If we want to display a graph of apdex values for
different
> >>> tolerated
> >>>>>>>>> times, than we would have to store all values, which
would
> >>>>>>>>> make
> >>> the
> >>>>>>>>> listener quite heavy on the memory side.
> >>>>>>>>>
> >>>>>>>>>> Thoughts ?
> >>>>>>>>>>
> >>>>>>>>> How will errors be counted? Sometimes I can cope
with errors,
> >>>>>>>>> as
> >>>>> long as
> >>>>>>>>> they are reported fast and sometimes an error would
result in
> >>>>>>>>> a
> >>> very
> >>>>>>>>> unhappy  consumer.
> >>>>>>>>>
> >>>>>>>> Good catch, I think only successful samples must be
taken into
> >>>>> account.
> >>>>>>>> Error will count only toward total requests (unsatisfied).
> >>>>>>>>
> >>>>>>>>> Regards
> >>>>>>>>>   Felix
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Cordialement.
> >>>>>>>> Philippe Mouawad.
> >>>>>>
> >>>>>> --
> >>>>>> Cordialement.
> >>>>>> Philippe Mouawad.
> >>>>
> >>>> --
> >>>> Cordialement.
> >>>> Philippe Mouawad.
> >>
> >> --
> >> Cordialement.
> >> Philippe Mouawad.
> >>
> >>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message