From dev-return-3901-apmail-jmeter-dev-archive=jmeter.apache.org@jmeter.apache.org Mon Feb 9 17:55:59 2015 Return-Path: X-Original-To: apmail-jmeter-dev-archive@minotaur.apache.org Delivered-To: apmail-jmeter-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 90EFF10143 for ; Mon, 9 Feb 2015 17:55:59 +0000 (UTC) Received: (qmail 54491 invoked by uid 500); 9 Feb 2015 17:55:59 -0000 Delivered-To: apmail-jmeter-dev-archive@jmeter.apache.org Received: (qmail 54464 invoked by uid 500); 9 Feb 2015 17:55:59 -0000 Mailing-List: contact dev-help@jmeter.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@jmeter.apache.org Delivered-To: mailing list dev@jmeter.apache.org Received: (qmail 54449 invoked by uid 99); 9 Feb 2015 17:55:59 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Feb 2015 17:55:59 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [81.169.162.220] (HELO h1611079.stratoserver.net) (81.169.162.220) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Feb 2015 17:55:54 +0000 Received: from [192.168.178.43] (dslb-088-077-207-107.088.077.pools.vodafone-ip.de [88.77.207.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by h1611079.stratoserver.net (Postfix) with ESMTPSA id 1C78D4948022 for ; Mon, 9 Feb 2015 18:54:58 +0100 (CET) Message-ID: <54D8F470.9080409@internetallee.de> Date: Mon, 09 Feb 2015 18:54:56 +0100 From: Felix Schumacher User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: dev@jmeter.apache.org Subject: Re: New Listener to generate HTML report at end of load test Was APDEX Computing and reporting References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Am 03.02.2015 um 13:38 schrieb Philippe Mouawad: > Hi, > Any additional opinions of dev team or users ? > I 'd like to know if I should continue efforts on this or stop it. I think it would be a nice feature to have a simple report at the end of a jmeter run. Maybe the user should ask for it by specifying a command line parameter like "--report". If this is integrated in the shell wrapper it could even be a program that is started after the jmeter run. But I don't think users will generally start another program themselves, so we shouldn't hide it too deep. Regards Felix > > Thanks > Regards > > On Monday, February 2, 2015, Philippe Mouawad > wrote: > >> >> On Monday, February 2, 2015, sebb > > wrote: >> >>> On 1 February 2015 at 22:41, Philippe Mouawad >>> wrote: >>>> On Sunday, February 1, 2015, sebb wrote: >>>> >>>>> On 1 February 2015 at 19:31, Philippe Mouawad >>>>> > wrote: >>>>>> Hi , >>>>>> I uploaded a screenshot and new patch code showing what I meant. >>>>>> I think including it in core should be considered seriously. >>>>> If the code reads the CSV file at the end of a test to produce the >>>>> HTML file, there is no need for it to be included in the JMeter code >>>>> which is used to run tests. >>>>> It should be a separate application. >>>>> >>>>> I don't share your opinion sebb. >>>> Having a report ready at end of load test is nowadays a standard, look >>> at >>>> market tools wether open source or not. >>>> I don't see the problem here, can you give more details ? >>> The problem is that it adds extra code to the JMeter application which >>> takes extra memory. >> >> My last implementation only uses memory at end of test not morz during it. >> >> >>> Also, if the user wants to process existing files, it will be easier >> to have a standalone application rather than having to start the >>> JMeter test application and then find the reporting tool. >> >> It is possible with it, just make a plan with 1 debug sampler and put >> listener. Report will be generated. >> >>> It would be useful if the tool could be used in batch mode to process >>> a set of CSV files. >> >> Maybe, but I think it's an additional feature. Also having another tool >> may lead to what happened to the report package, it seems it was never >> used. But if you implement it it will be ok for me. >> >> >>> It's also likely to be easier for others to contribute additional >>> reporting tools if they are part of a separate application. >> I don't understand this ? how is it easier if it's in jmeter core. If it's >> apart I don't support this option. >> >> >> >>> It should make the coding requirements simpler. >> how ? >> >>>> Having another application( by the way do you mean provided as a tool in >>>> jmeter or third party) means in the first case you need to setup another >>>> tool and in the latter case need to develop your own, or use or pay one. >>> Of course the code should be included with JMeter. >>> But it should be a separate tool. >>> >>>> This is the major drawback of JMeter I hear from customers and read >>> around >>>> the net. >>>> I first though it would not be that easy but it appears we are able to >>>> reuse existing code. >>> I don't follow. >>> >>>> If you disagree, I think we should wait for other team members their >>>> opinion and/or setup a vote for this. >>> The main JMeter code should be reserved for setting up and running tests. >>> >> IMHO reporting must be part of JMeter, and anyway it is already through >> some Listeners , which although not perfect bring useful infos. The aim of >> this new listener is: >> 1/ Allow generation in gui and non gui mode >> 2/ Have a "sexy" (not yet ) report readable in browser with dynamic >> behaviour (zoom, select some samples...) >> 3/ Allow generation at end of load test >> 4/ Make it easily customizable as FTL will be a property >> 5/ Make it extensible in the future >> >> >> >> >>>> >>>> >>>> >>>>>> @Felix thanks for review of initial patch. >>>>>> Regards >>>>>> >>>>>> On Sunday, February 1, 2015, sebb > >>>>> wrote: >>>>>>> On 31 January 2015 at 13:16, Philippe Mouawad >>>>>>> > wrote: >>>>>>>> On Sat, Jan 31, 2015 at 12:30 PM, Felix Schumacher < >>>>>>>> felix.schumacher@internetallee.de > >>>>> wrote: >>>>>>>>> Am 30.01.2015 um 13:31 schrieb Philippe Mouawad: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> I intend to commit a BackendListener client implementation that >>>>>>> computes >>>>>>>>>> APDEX at end of Load Test. >>>>>>>>>> It will take: >>>>>>>>>> >>>>>>>>>> - Acceptable Response Time (http://www.apdex.fr/) T >>>>>>>>>> >>>>>>>>> The link was in french (my french is not very good (really non >>>>> existant >>>>>>>>> :)), but luckily wikipedia had an article about apdex, which I >>> could >>>>>>> read. >>>>>>>> Sorry for the french link some references: >>>>>>>> http://www.apdex.org/overview.html >>>>>>>> >>>>>>>> - Compute F as 4xT but allow customization >>>>>>>>>> - List of samples taken into account >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This listener will compute it during load test and generate an >>> HTML >>>>>>> file >>>>>>>>>> at >>>>>>>>>> end of test containing it. >>>>>>>>>> >>>>>>>>>> This listener could be later enhanced to : >>>>>>>>>> >>>>>>>>>> - Generate equivalent of Aggregate Report >>>>>>>>>> - Possibly graphs based on chartjs or some other graphing JS >>>>>>> Library >>>>>>>>> I understand, that apdex is a simple mean to have one metric. As >>>>> such I >>>>>>>>> wonder why we would want to graph it. >>>>>>>> In fact my proposal was larger than just APDEX. >>>>>>>> What I am proposing is to implement a report generator that at >>> end of >>>>>>> Test >>>>>>>> would compute HTML page with: >>>>>>>> - APDEX >>>>>>>> - Some graphs like: >>>>>>>> - Plot of OK/KO (http://www.chartjs.org/docs/#doughnut-pie-chart >>> ) >>>>>>>> - Request Summary (what's currently in Aggregate Graph) >>>>>>>> http://www.chartjs.org/docs/#bar-chart >>>>>>>> - Response Time over time ( >>> http://www.chartjs.org/docs/#line-chart) >>>>>>>> I think all this should in fact be computed at end of load test >>>>> instead >>>>>>> of >>>>>>>> during it. >>>>>>>> The input of this listener would be the generated CSV file, and at >>>>> end of >>>>>>>> Test it would read it an compute all this. >>>>>>> In which case it should be a stand-alone app. >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> If we want to display a graph of apdex values for different >>> tolerated >>>>>>>>> times, than we would have to store all values, which would make >>> the >>>>>>>>> listener quite heavy on the memory side. >>>>>>>>> >>>>>>>>>> Thoughts ? >>>>>>>>>> >>>>>>>>> How will errors be counted? Sometimes I can cope with errors, as >>>>> long as >>>>>>>>> they are reported fast and sometimes an error would result in a >>> very >>>>>>>>> unhappy consumer. >>>>>>>>> >>>>>>>> Good catch, I think only successful samples must be taken into >>>>> account. >>>>>>>> Error will count only toward total requests (unsatisfied). >>>>>>>> >>>>>>>>> Regards >>>>>>>>> Felix >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Cordialement. >>>>>>>> Philippe Mouawad. >>>>>> >>>>>> -- >>>>>> Cordialement. >>>>>> Philippe Mouawad. >>>> >>>> -- >>>> Cordialement. >>>> Philippe Mouawad. >> >> -- >> Cordialement. >> Philippe Mouawad. >> >> >> >>