esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Kohler <markus.koh...@gmail.com>
Subject Re: Performance Tests - Suggestions and Ideas
Date Thu, 19 Nov 2009 08:12:11 GMT
Hi all,
See my comments below...

On Thu, Nov 19, 2009 at 8:27 AM, Richard Hirsch <hirsch.dick@gmail.com>wrote:

> I enhanced the performance test page in the wiki
> (http://cwiki.apache.org/confluence/display/ESME/Performance+tests)
> with more details and test ideas
>
>
I like that.


> @Markus: I think it would great if you could do the performance tests
> once a week on the Stax performance instance.  Initially, it would be
> great just to have test suite that we can run. It is critical that we
> can put such tests into our continuous integration cycle but until
> this is possible, it would be great if you could just do these short
> tests on a weekly basis. That would allow us to make sure that code
> changes aren't impacting our performance. I'd like to make these tests
> as easy for you to perform as possible.
>
>
The tests could also be used as simple functional tests. They would have
catched for example the javascript problem that we had lately.


> Although regression testing would be one goal, the changing of small
> parameters (adding one action for each user) and seeing the impact on
> performance would also be very interesting.
>
> Yes. I certainly need some simple extensions to what I've done so far. For
example users should follow other users.
I think this way we could even manually do some simple performance
tests/analysis.


> We should probably define some sort of a process that describes the
> tests to make sure that the basis is always the same. For example,
>
> 1. Drop existing DB - this can be via a stax command
>

Yes for the regression test that is a must.


> 2. Create existing DB  - this can be via a stax command
>

Not sure, what you mean. Can we switch from one DB to another, for example
with different data sets (nr. of Users)?
That would be fantastic.


> 3. Deploy application - - this can be via a stax command
> 4. Test - this could be done via maven
> 5. Take screenshots of Stax console (still have to define which
> screens we need - maybe like the ones you sent me last time)
>

Stax doesn't have a way to get the data as a file (CSV or whatever)?
I could use Selenium to navigate to the Stax console at the end of the test
and do some screenshots automatically.


> 6. Mail all the details to me and I post to wiki
>
>
Yes. I can do that.

My plan is to get the low hanging fruits first of course.
Those are (at the top of my head):
- Checking the client side. E.g. cache settings, request compression etc. .
Not sure whether I should do it now, because we will get a new UI hopefully
soon?
- memory usage analysis. This is low hanging, because I've done it so often,
I could do it while sleeping ;)
- quick memory allocation tracing analysis. E.g. how much memory is
allocated and reclaimed for one interaction step. Can be done manually, btw
does someone know how to tell maven "jetty:run" which JVM to use?

Regards,
Markus


> Thoughts?
>
> D.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message