jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Schumacher <felix.schumac...@internetallee.de>
Subject Re: New Listener to generate HTML report at end of load test Was APDEX Computing and reporting
Date Mon, 09 Feb 2015 17:54:56 GMT
Am 03.02.2015 um 13:38 schrieb Philippe Mouawad:
> Hi,
> Any additional opinions of dev team or users ?
> I 'd like to know if I should continue efforts on this or stop it.
I think it would be a nice feature to have a simple report at the end of 
a jmeter run. Maybe the user should ask for it by specifying a command 
line parameter like "--report".

If this is integrated in the shell wrapper it could even be a program 
that is started after the jmeter run. But I don't think users will 
generally start another program themselves, so we shouldn't hide it too 
deep.

Regards
  Felix
>
> Thanks
> Regards
>
> On Monday, February 2, 2015, Philippe Mouawad <philippe.mouawad@gmail.com>
> wrote:
>
>>
>> On Monday, February 2, 2015, sebb <sebbaz@gmail.com
>> <javascript:_e(%7B%7D,'cvml','sebbaz@gmail.com');>> wrote:
>>
>>> On 1 February 2015 at 22:41, Philippe Mouawad
>>> <philippe.mouawad@gmail.com> wrote:
>>>> On Sunday, February 1, 2015, sebb <sebbaz@gmail.com> wrote:
>>>>
>>>>> On 1 February 2015 at 19:31, Philippe Mouawad
>>>>> <philippe.mouawad@gmail.com <javascript:;>> wrote:
>>>>>> Hi ,
>>>>>> I uploaded a screenshot and new patch code showing what I meant.
>>>>>> I think including it in core should be considered seriously.
>>>>> If the code reads the CSV file at the end of a test to produce the
>>>>> HTML file, there is no need for it to be included in the JMeter code
>>>>> which is used to run tests.
>>>>> It should be a separate application.
>>>>>
>>>>> I don't share your opinion sebb.
>>>> Having a report ready at end of load test is nowadays a standard, look
>>> at
>>>> market tools wether open source or not.
>>>> I don't see the problem here, can you give more details ?
>>> The problem is that it adds extra code to the JMeter application which
>>> takes extra memory.
>>
>> My last implementation only uses memory at end of test not morz during it.
>>
>>
>>> Also, if the user wants to process existing files, it will be easier
>> to have a standalone application rather than having to start the
>>> JMeter test application and then find the reporting tool.
>>
>> It is possible with it, just make a plan with 1 debug sampler and put
>> listener. Report will be generated.
>>
>>> It would be useful if the tool could be used in batch mode to process
>>> a set of CSV files.
>>
>> Maybe, but I think it's an additional feature. Also having another tool
>> may lead to what happened to the report package, it seems it was never
>> used.  But if you implement it it will be ok for me.
>>
>>
>>> It's also likely to be easier for others to contribute additional
>>> reporting tools if they are part of a separate application.
>> I don't understand this ? how is it easier if it's in jmeter core. If it's
>> apart I don't support this option.
>>
>>
>>
>>> It should make the coding requirements simpler.
>> how ?
>>
>>>> Having another application( by the way do you mean provided as a tool in
>>>> jmeter or third party) means in the first case you need to setup another
>>>> tool and in the latter case need to develop your own, or use or pay one.
>>> Of course the code should be included with JMeter.
>>> But it should be a separate tool.
>>>
>>>> This is the major drawback of JMeter I hear from customers and read
>>> around
>>>> the net.
>>>> I first though it would not be that easy but it appears we are able to
>>>> reuse existing code.
>>> I don't follow.
>>>
>>>> If you disagree, I think we should wait for other team members their
>>>> opinion and/or setup a vote for this.
>>> The main JMeter code should be reserved for setting up and running tests.
>>>
>> IMHO reporting must be part of JMeter, and anyway it is already through
>> some Listeners , which although not perfect bring useful infos. The aim of
>> this new listener is:
>> 1/ Allow generation in gui and non gui mode
>> 2/ Have a "sexy" (not yet ) report readable in browser with dynamic
>> behaviour (zoom, select some samples...)
>> 3/ Allow generation at end of load test
>> 4/ Make it easily customizable as FTL will be a property
>> 5/ Make it extensible in the future
>>
>>
>>
>>
>>>>
>>>>
>>>>
>>>>>> @Felix thanks for review of initial patch.
>>>>>> Regards
>>>>>>
>>>>>> On Sunday, February 1, 2015, sebb <sebbaz@gmail.com <javascript:;>>
>>>>> wrote:
>>>>>>> On 31 January 2015 at 13:16, Philippe Mouawad
>>>>>>> <philippe.mouawad@gmail.com <javascript:;> <javascript:;>>
wrote:
>>>>>>>> On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher <
>>>>>>>> felix.schumacher@internetallee.de <javascript:;> <javascript:;>>
>>>>> wrote:
>>>>>>>>> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I intend to commit a BackendListener client implementation
that
>>>>>>> computes
>>>>>>>>>> APDEX at end of Load Test.
>>>>>>>>>> It will take:
>>>>>>>>>>
>>>>>>>>>>      - Acceptable Response Time (http://www.apdex.fr/)
T
>>>>>>>>>>
>>>>>>>>> The link was in french (my french is not very good (really
non
>>>>> existant
>>>>>>>>> :)), but luckily wikipedia had an article about apdex,
which I
>>> could
>>>>>>> read.
>>>>>>>> Sorry for the french link some references:
>>>>>>>> http://www.apdex.org/overview.html
>>>>>>>>
>>>>>>>>       - Compute F as 4xT but allow customization
>>>>>>>>>>      - List of samples taken into account
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This listener will compute it during load test and
generate an
>>> HTML
>>>>>>> file
>>>>>>>>>> at
>>>>>>>>>> end of test containing it.
>>>>>>>>>>
>>>>>>>>>> This listener could be later enhanced to :
>>>>>>>>>>
>>>>>>>>>>      - Generate equivalent of Aggregate Report
>>>>>>>>>>      - Possibly graphs based on chartjs or some other
graphing JS
>>>>>>> Library
>>>>>>>>> I understand, that apdex is a simple mean to have one
metric. As
>>>>> such I
>>>>>>>>> wonder why we would want to graph it.
>>>>>>>> In fact my proposal was larger than just APDEX.
>>>>>>>> What I am proposing is to implement a report generator that
at
>>> end of
>>>>>>> Test
>>>>>>>> would compute HTML page with:
>>>>>>>> - APDEX
>>>>>>>> - Some graphs like:
>>>>>>>> - Plot of OK/KO  (http://www.chartjs.org/docs/#doughnut-pie-chart
>>> )
>>>>>>>> - Request Summary (what's currently in Aggregate Graph)
>>>>>>>> http://www.chartjs.org/docs/#bar-chart
>>>>>>>> - Response Time over time (
>>> http://www.chartjs.org/docs/#line-chart)
>>>>>>>> I think all this should in fact be computed at end of load
test
>>>>> instead
>>>>>>> of
>>>>>>>> during it.
>>>>>>>> The input of this listener would be the generated CSV file,
and at
>>>>> end of
>>>>>>>> Test it would read it an compute all this.
>>>>>>> In which case it should be a stand-alone app.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> If we want to display a graph of apdex values for different
>>> tolerated
>>>>>>>>> times, than we would have to store all values, which
would make
>>> the
>>>>>>>>> listener quite heavy on the memory side.
>>>>>>>>>
>>>>>>>>>> Thoughts ?
>>>>>>>>>>
>>>>>>>>> How will errors be counted? Sometimes I can cope with
errors, as
>>>>> long as
>>>>>>>>> they are reported fast and sometimes an error would result
in a
>>> very
>>>>>>>>> unhappy  consumer.
>>>>>>>>>
>>>>>>>> Good catch, I think only successful samples must be taken
into
>>>>> account.
>>>>>>>> Error will count only toward total requests (unsatisfied).
>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>>   Felix
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Cordialement.
>>>>>>>> Philippe Mouawad.
>>>>>>
>>>>>> --
>>>>>> Cordialement.
>>>>>> Philippe Mouawad.
>>>>
>>>> --
>>>> Cordialement.
>>>> Philippe Mouawad.
>>
>> --
>> Cordialement.
>> Philippe Mouawad.
>>
>>
>>
>>


Mime
View raw message