incubator-esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Hirsch <hirsch.d...@gmail.com>
Subject Re: Performance Tests - Suggestions and Ideas
Date Thu, 19 Nov 2009 08:22:32 GMT
See my comments below.

On Thu, Nov 19, 2009 at 9:12 AM, Markus Kohler <markus.kohler@gmail.com> wrote:
> Hi all,
> See my comments below...
>
> On Thu, Nov 19, 2009 at 8:27 AM, Richard Hirsch <hirsch.dick@gmail.com>wrote:
>
>> I enhanced the performance test page in the wiki
>> (http://cwiki.apache.org/confluence/display/ESME/Performance+tests)
>> with more details and test ideas
>>
>>
> I like that.
>
>
>> @Markus: I think it would great if you could do the performance tests
>> once a week on the Stax performance instance.  Initially, it would be
>> great just to have test suite that we can run. It is critical that we
>> can put such tests into our continuous integration cycle but until
>> this is possible, it would be great if you could just do these short
>> tests on a weekly basis. That would allow us to make sure that code
>> changes aren't impacting our performance. I'd like to make these tests
>> as easy for you to perform as possible.
>>
>>
> The tests could also be used as simple functional tests. They would have
> catched for example the javascript problem that we had lately.
>
>
>> Although regression testing would be one goal, the changing of small
>> parameters (adding one action for each user) and seeing the impact on
>> performance would also be very interesting.
>>
>> Yes. I certainly need some simple extensions to what I've done so far. For
> example users should follow other users.
> I think this way we could even manually do some simple performance
> tests/analysis.
>
>
>> We should probably define some sort of a process that describes the
>> tests to make sure that the basis is always the same. For example,
>>
>> 1. Drop existing DB - this can be via a stax command
>>
>
> Yes for the regression test that is a must.
>
>
>> 2. Create existing DB  - this can be via a stax command
>>
> Not sure, what you mean. Can we switch from one DB to another, for example
> with different data sets (nr. of Users)?
> That would be fantastic.

You can only do this at deplyoment time. By changing some stax
configuration files. No problem but only at deployment and
not-run-time

>
>
>> 3. Deploy application - - this can be via a stax command
>> 4. Test - this could be done via maven
>> 5. Take screenshots of Stax console (still have to define which
>> screens we need - maybe like the ones you sent me last time)
>>
>
> Stax doesn't have a way to get the data as a file (CSV or whatever)?


You can access the mysql database via jdbc - there are details on the
stax console page. This is probably the best way to do it. I didn't
have access to these ports, because of corporate firewall
restrictions.

> I could use Selenium to navigate to the Stax console at the end of the test
> and do some screenshots automatically.

That would be very cool.

>
>
>> 6. Mail all the details to me and I post to wiki
>>
>>
> Yes. I can do that.
>
> My plan is to get the low hanging fruits first of course.
> Those are (at the top of my head):
> - Checking the client side. E.g. cache settings, request compression etc. .
> Not sure whether I should do it now, because we will get a new UI hopefully
> soon?
> - memory usage analysis. This is low hanging, because I've done it so often,
> I could do it while sleeping ;)
> - quick memory allocation tracing analysis. E.g. how much memory is
> allocated and reclaimed for one interaction step. Can be done manually, btw
> does someone know how to tell maven "jetty:run" which JVM to use?
>

Added these ideas to the wiki page

> Regards,
> Markus
>
>
>> Thoughts?
>>
>> D.
>>
>

Mime
View raw message