jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antonio Gomes Rodrigues <ra0...@gmail.com>
Subject Re: New Listener to generate HTML report at end of load test Was APDEX Computing and reporting
Date Tue, 03 Feb 2015 13:03:12 GMT
Hi,

For your information, APM tools use APDEX (and a little more) in UEM
features
And customers like it (easy to make some great dashboard)
And I think it will be great to have it in JMeter

Antonio
2015-02-03 13:38 GMT+01:00 Philippe Mouawad <philippe.mouawad@gmail.com>:

> Hi,
> Any additional opinions of dev team or users ?
> I 'd like to know if I should continue efforts on this or stop it.
>
> Thanks
> Regards
>
> On Monday, February 2, 2015, Philippe Mouawad <philippe.mouawad@gmail.com>
> wrote:
>
> >
> >
> > On Monday, February 2, 2015, sebb <sebbaz@gmail.com
> > <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
> >
> >> On 1 February 2015 at 22:41, Philippe Mouawad
> >> <philippe.mouawad@gmail.com> wrote:
> >> > On Sunday, February 1, 2015, sebb <sebbaz@gmail.com> wrote:
> >> >
> >> >> On 1 February 2015 at 19:31, Philippe Mouawad
> >> >> <philippe.mouawad@gmail.com <javascript:;>> wrote:
> >> >> > Hi ,
> >> >> > I uploaded a screenshot and new patch code showing what I meant.
> >> >> > I think including it in core should be considered seriously.
> >> >>
> >> >> If the code reads the CSV file at the end of a test to produce the
> >> >> HTML file, there is no need for it to be included in the JMeter code
> >> >> which is used to run tests.
> >> >> It should be a separate application.
> >> >>
> >> >> I don't share your opinion sebb.
> >> > Having a report ready at end of load test is nowadays a standard, look
> >> at
> >> > market tools wether open source or not.
> >> > I don't see the problem here, can you give more details ?
> >>
> >> The problem is that it adds extra code to the JMeter application which
> >> takes extra memory.
> >
> >
> > My last implementation only uses memory at end of test not morz during
> it.
> >
> >
> >> Also, if the user wants to process existing files, it will be easier
> >
> > to have a standalone application rather than having to start the
> >> JMeter test application and then find the reporting tool.
> >
> >
> > It is possible with it, just make a plan with 1 debug sampler and put
> > listener. Report will be generated.
> >
> >>
> >> It would be useful if the tool could be used in batch mode to process
> >> a set of CSV files.
> >
> >
> > Maybe, but I think it's an additional feature. Also having another tool
> > may lead to what happened to the report package, it seems it was never
> > used.  But if you implement it it will be ok for me.
> >
> >
> >>
> >> It's also likely to be easier for others to contribute additional
> >> reporting tools if they are part of a separate application.
> >
> > I don't understand this ? how is it easier if it's in jmeter core. If
> it's
> > apart I don't support this option.
> >
> >
> >
> >> It should make the coding requirements simpler.
> >
> > how ?
> >
> >>
> >> > Having another application( by the way do you mean provided as a tool
> in
> >> > jmeter or third party) means in the first case you need to setup
> another
> >> > tool and in the latter case need to develop your own, or use or pay
> one.
> >>
> >> Of course the code should be included with JMeter.
> >> But it should be a separate tool.
> >>
> >> > This is the major drawback of JMeter I hear from customers and read
> >> around
> >> > the net.
> >>
> >> > I first though it would not be that easy but it appears we are able to
> >> > reuse existing code.
> >>
> >> I don't follow.
> >>
> >> > If you disagree, I think we should wait for other team members their
> >> > opinion and/or setup a vote for this.
> >>
> >> The main JMeter code should be reserved for setting up and running
> tests.
> >>
> > IMHO reporting must be part of JMeter, and anyway it is already through
> > some Listeners , which although not perfect bring useful infos. The aim
> of
> > this new listener is:
> > 1/ Allow generation in gui and non gui mode
> > 2/ Have a "sexy" (not yet ) report readable in browser with dynamic
> > behaviour (zoom, select some samples...)
> > 3/ Allow generation at end of load test
> > 4/ Make it easily customizable as FTL will be a property
> > 5/ Make it extensible in the future
> >
> >
> >
> >
> >> >
> >> >
> >> >
> >> >
> >> >> > @Felix thanks for review of initial patch.
> >> >> > Regards
> >> >> >
> >> >> > On Sunday, February 1, 2015, sebb <sebbaz@gmail.com
> <javascript:;>>
> >> >> wrote:
> >> >> >
> >> >> >> On 31 January 2015 at 13:16, Philippe Mouawad
> >> >> >> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>>
wrote:
> >> >> >> > On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
> >> >> >> > felix.schumacher@internetallee.de <javascript:;>
> <javascript:;>>
> >> >> wrote:
> >> >> >> >
> >> >> >> >> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
> >> >> >> >>
> >> >> >> >>> Hi,
> >> >> >> >>> I intend to commit a BackendListener client implementation
> that
> >> >> >> computes
> >> >> >> >>> APDEX at end of Load Test.
> >> >> >> >>> It will take:
> >> >> >> >>>
> >> >> >> >>>     - Acceptable Response Time (http://www.apdex.fr/)
T
> >> >> >> >>>
> >> >> >> >> The link was in french (my french is not very good
(really non
> >> >> existant
> >> >> >> >> :)), but luckily wikipedia had an article about apdex,
which I
> >> could
> >> >> >> read.
> >> >> >> >>
> >> >> >> >
> >> >> >> > Sorry for the french link some references:
> >> >> >> > http://www.apdex.org/overview.html
> >> >> >> >
> >> >> >> >      - Compute F as 4xT but allow customization
> >> >> >> >>>     - List of samples taken into account
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>> This listener will compute it during load test
and generate an
> >> HTML
> >> >> >> file
> >> >> >> >>> at
> >> >> >> >>> end of test containing it.
> >> >> >> >>>
> >> >> >> >>> This listener could be later enhanced to :
> >> >> >> >>>
> >> >> >> >>>     - Generate equivalent of Aggregate Report
> >> >> >> >>>     - Possibly graphs based on chartjs or some
other graphing
> JS
> >> >> >> Library
> >> >> >> >>>
> >> >> >> >> I understand, that apdex is a simple mean to have
one metric.
> As
> >> >> such I
> >> >> >> >> wonder why we would want to graph it.
> >> >> >> >
> >> >> >> > In fact my proposal was larger than just APDEX.
> >> >> >> > What I am proposing is to implement a report generator
that at
> >> end of
> >> >> >> Test
> >> >> >> > would compute HTML page with:
> >> >> >> > - APDEX
> >> >> >> > - Some graphs like:
> >> >> >> > - Plot of OK/KO  (
> http://www.chartjs.org/docs/#doughnut-pie-chart
> >> )
> >> >> >> > - Request Summary (what's currently in Aggregate Graph)
> >> >> >> > http://www.chartjs.org/docs/#bar-chart
> >> >> >> > - Response Time over time (
> >> http://www.chartjs.org/docs/#line-chart)
> >> >> >> >
> >> >> >> > I think all this should in fact be computed at end of
load test
> >> >> instead
> >> >> >> of
> >> >> >> > during it.
> >> >> >> > The input of this listener would be the generated CSV
file, and
> at
> >> >> end of
> >> >> >> > Test it would read it an compute all this.
> >> >> >>
> >> >> >> In which case it should be a stand-alone app.
> >> >> >>
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >> If we want to display a graph of apdex values for
different
> >> tolerated
> >> >> >> >> times, than we would have to store all values, which
would make
> >> the
> >> >> >> >> listener quite heavy on the memory side.
> >> >> >> >>
> >> >> >> >>>
> >> >> >> >>> Thoughts ?
> >> >> >> >>>
> >> >> >> >> How will errors be counted? Sometimes I can cope
with errors,
> as
> >> >> long as
> >> >> >> >> they are reported fast and sometimes an error would
result in a
> >> very
> >> >> >> >> unhappy  consumer.
> >> >> >> >>
> >> >> >> > Good catch, I think only successful samples must be taken
into
> >> >> account.
> >> >> >> > Error will count only toward total requests (unsatisfied).
> >> >> >> >
> >> >> >> >>
> >> >> >> >> Regards
> >> >> >> >>  Felix
> >> >> >> >>
> >> >> >> >>
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > Cordialement.
> >> >> >> > Philippe Mouawad.
> >> >> >>
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Cordialement.
> >> >> > Philippe Mouawad.
> >> >>
> >> >
> >> >
> >> > --
> >> > Cordialement.
> >> > Philippe Mouawad.
> >>
> >
> >
> > --
> > Cordialement.
> > Philippe Mouawad.
> >
> >
> >
> >
>
> --
> Cordialement.
> Philippe Mouawad.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message