jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philippe Mouawad <philippe.moua...@gmail.com>
Subject Re: Graphite Listener
Date Wed, 01 Jan 2014 15:08:55 GMT
On Tue, Dec 31, 2013 at 9:00 PM, Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:

> >> I believe JMeter plugin-like interface between injector and performance
> >> repository would enable live monitoring of sytem under test.
> sebb>Exactly. The JMeter architecture is designed to allow for easy
> integration of 3rd party plugins.
>
> I need to double-check, however I did not manage to specify "path to
> plugins.jar folder" via command line option.
> Ultimately I would love to have JMeter installation and plugins in
> completely separate folders. That simplifies "jmeter upgrade", "check
> what plugins are installed", "compose test harness from maven"
> usecases.
>
> Add this to user.properties:
plugin_dependency_paths which will contains the path to third party jar

user.classpath which will contain path to dependencies of third party jar



> Are there "asynchronous output" interfaces from JMeter?
>

in remote testing you are in async mode with:
mode=StrippedAsynch
or
mode= Asynch

>
> Is there a way to send listener result via regular samplers?
>

Why would you like to do that ?

sebb> Proper analysis should take place offline after the test has
> completed.
> Very true.
> However, it is quite important to perform online analysis to be able
> to adjust the test.
> Say, adjust the load, fix bugs in script, correct system configuration,
> etc.
>
> >> One can parse raw csv/xml results and upload for the analysis, however
> it
> >> is likely to create big latency gap between collection and the
> >> visualization.
> >
> sebb> Is that really a problem for most users?
> How do we measure that?
>

A vote would be a way to sort it out.

>
> Here are the most relevant scenarios for our company:
> 1) Durability testing. Say, you launch 1..7days long test script.
> It is crucial to know if the system is yet stable.
> That includes multiple KPIs (throughput, latency, failrate%, memory
> consumption, gc, cpu%), and the request processing KPIs are not the
> least important ones.
> In case JMeter sampler info is not ready until the whole test is
> finished it is major drawback.
>

Agree

>
> 2) Automated scalability testing. Say you want to identify maximum
> load the system will sustain. One way to identify it is to gradually
> increase the load and see if the system is stable (e.g. queues do not
> build up, failrate is 0, response times are stable, etc).
> Having data displayed in near-realtime helps a lot.
> Especially, when you run test suite in different environment (e.g.
> acceptance of new software/hardware)
>

Agree

>
> 3) Tight-schedule testing. When performing load testing at customer
> environment (e.g. acceptance of production environment), it is
> important to make informed decisions. Is good to see if your test
> works as expected when the test is running, not when you've done 4
> hours of testing and analyzing afterwards.
>

Agree

>
> 4) Red-green detection during regular 30-60min testing. Our scenarios
> involve multiple jmx scripts and lots of samples.
> We use JMeter GUI only for script development. We use just console for
> load testing to avoid injector slowness/out of memory/etc.
> Currently it is hard to tell if the test goes as expected: failrate%,
> number of samples, response times, etc.
>
> Agree


> sebb> That would be a sensible addition to JMeter to allow performance data
> > to be readily saved to an arbitrary repo.
> True.
> OOB samplers might work (e.g. http sampler) might work for the "output
> interfaces".
> For instance "http sampler" under "stats listener" posts the result of
> "stats listener" to the configured end-point.
>
> However, I believe this kind of integration should be asynchronous to
> avoid impact result collection on the test scenario. If trying to use
> sync result posting, we could result into multi-second test hang due
> to lag of performance repository receiver.
>
> Currently the Graphite Listener patch it is implemented this way:
1) Synchronously, success , failures, min and max are updated (very fast
computations), percentile related data is also computed (Looking at code of
commons-math3 I don't see why it would not perform fine). If  further tests
show the percentile computation is costly , then we could delay this to the
2) thread below. But the tests I did did not show problem.
2) And in the background, a task runs every minute to send current computed
results to Graphite server , resetting datas except for percentile related
ones.



> Regards,
> Vladimir Sitnikov
>



-- 
Cordialement.
Philippe Mouawad.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message