jmeter-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emilian Bold <emilian.b...@protonmail.ch>
Subject Some JMeter (technical) concerns
Date Mon, 30 Oct 2017 12:27:15 GMT
Hello,

I've been following JMeter for a while now, even did a separate project based on JMeter (YaMeter.com
) so I figured I should mention some (technical) concerns of mine about JMeter.

I thought about sending multiple emails, each about a single aspect, but I will just list
everything here and people are invited to only answer the subset they care about.

1. Userbase

It seems to me there are no clear metrics about the JMeter userbase. We have no phone-home,
no plugin portal, no download statistics(!?) so we can only use indirect means of estimating
the userbase: stackoverflow questions and such.

This seems a very poor state of affairs. Resources are scarce: how do you know what to focus
on if you have very little insight into who your users are, what are they doing and where
are they struggling?

1.1 Metrics

I believe it is possible to add usage metrics to JMeter while respecting the ASF policy on
privacy, etc.

It's also unclear to me how download stats are not shared by the Apache mirrors (any ASF document
explaining this policy?).

2. Speed of development

Compared to some of the plugins, JMeter seems to move very slowly. This might create the impression
of stability but I believe it's just a sign of being resource constrained.

Are any companies putting resources into JMeter? I see a lot of companies / startups trying
to make money off JMeter but how many are contributing back?

BTW, one reason I decided to do YaMeter.com instead of trying to push into JMeter proper is
the approval speed. Getting a patch in seems to take a lot of effort and I can only imagine
how long it would have take to try something as large as what I did with YaMeter.

3. UI

The JMeter UI needs some improving. The problem is not that it uses Java / Swing because even
Swing can be made to look good but we need people to look into this. Is it "good enough" for
people being forced to use JMeter in a corporate setting? Would an user pick it just based
on the UI?

3.1 Swing

If Swing remains the future of JMeter, I suggest looking into a better framework like Apache
NetBeans Platform which provides a lot of useful things for desktop Java apps.

Generally speaking, it would be good to split the UI from the "core" so we have some path
where we evolve to some other UI.

3.2 Web based

Alternatively, a web-based interface could be made for JMeter. Any interest in this?

Just like companies use local GitLab installations, I am certain a lot of companies would
install a local Apache JMeter Web Portal where all the load testing would be centralised.

4. Plugin Manager

It seems unacceptable to me that JMeter does not have a Plugin Manager and a plugin portal
with some lower-usage plugins (eg. mongodb).

It's acceptable for other plugins to exist elsewhere, but those should be just another "PPA"
(see Ubuntu PPA) or another "plugin repository" URL to register.

5. Out of the box experience

The way I see it, Apache JMeter is not just a "platform" with some core files and external
plugins but something usable out of the box.

So, except niche situations, users should be catered by the default JMeter install.

5.1 HTTP2

A major problem being the lack of an official HTTP2 plugin, which people have to take from
elsewhere. Blazemeter is probably quite happy with their implementation being used since it
funnels people into their services.

6. Project based

I find the whole configuration .properties file a very poor solution to the notion of a "project".
If a JMX needs a .properties change, then those two belong together.

JMeter should have the notion of a 'project' and maybe even be capable of having two projects
open at once. And executing two unrelated tests at once. (Same for servers perhaps?)

6.1 Too many .properties

I also believe JMeter tries to be too configurable sometimes with flags for everything.

7. Module system

JMeter is using a lot of reflection to reinvent its own quirky module system (which provides
the way the plugins work).

This needs some standardization work.

The NetBeans Platform, for example, comes with its own module system that also supports OSGI
bundles (another module system). Then, there's the JVM service locator (for services) and
the Java 9 modules.

8. Remote is using old technologies

RMI is a killer in modern setups. It needs open ports on both the slave and master for communication
which only makes sense within the same intranet and it's a pain to configure when using Cloud
resources.

The whole communication between JMeter client and servers should be rewritten using more modern
protocols.

9. Cloud aware

The notion that the user tries to configure N servers then gives JMeter a list of IPs for
remote load testing seems... primitive. This is something I've been trying to solve with YaMeter.com

10. Packaging

JMeter should be everywhere. Official Docker files, AMI images, etc.

11. Pipeline aware / agents

JMeter should think more about making it easy to be hooked up into a CI / CD pipeline.

12. Script files instead of JMX

The XML files are a pain to diff and people like to track / compare changes in modern devops
shops.

Maybe a switch to YAML might suffice, but I believe a new scripting support (based on Groovy
or whatever) should be developed.

--emi

Mime
View raw message