Jordi Salvat i Alabart wrote:
> My personal position would be to move towards allowing functional
> testing, but still keep performance testing a priority. That is: never
> make a change that improves functional testing ability if it degrades
> performance testing ability. The opposite could happen, although I
> guess we would always go great lengths to try to make both
> requirements compatible.
>
> The rationale for this is:
> - JMeter is a performance testing tool -- or at least that's its charter.
> - Performance testing requires functional testing features: you need
> to see that function is not degraded under load.
>
This makes a lot of sense.
The functional testing in JMeter seems OK even now, but performance and
functional testing are very different.
A performance test plan may have as few as 10 or less requests repeated
in loops and across threads.
The tester doesn't mind working a bit harder to set up each request to
keep the client-side load light.
(eg using ugly __regexFunction syntax to send a hidden parameter from
last response. -
${__regexFunction(<input type='hidden' name='someID'
value='(.*)'>,$1$,1,,,)}
Note also how brittle this is (case-sensitive, expects single-quotes
and exact whitespace,and doesn't recognize comments or other false
positives)
)
A Functional test plan for a web app may easily have 1000's or more
requests testing every possible regression failure point.
Here the client-side speed is a small factor.
The human effort required for creation/Maintenance and the brittleness
of the tests are much more important.
(in example above, you would expect a good HTTP Functional test app to
get hidden parameters with some much simpler function -
eg getHiddenParameter("someID")
- you would also expect this to have parsed the HTML so it can eg
ignore comments or other unwanted text that simple req exp would match.)
Also the level of technical expertise for the tester writing the plan
can be very different.
Having said that, the pluggable architecture of JMeter may allow for
"Clever but Slow" Components to be made that are useful for Functional
Testing,
but should not be installed in performance tests?
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
|