On 6/28/06, Alex Karasulu <firstname.lastname@example.org
Need Benchmarking/Profiling Perf Suite (BPPS) for ApacheDS
I did the first thing anyone would do. I tapped Emmanuel on the
shoulder to ask him for his materials for his AC EU presentation. I did
not want to repeat the work that he had already done.
Sorry about me being in a rush, I provided very few information, I'm afraid :(
Please Emmanuel take no offense but I found the setup and repeated work
to be a bit of a hassle.
I find it a PITA :)
I'm sure you were bothered by doing things
Sometime lack of time drive you to make the most common mistake : not spending some more time doing things correct immediatly, instead of postponing this task forever...
Plus I wanted to profile these tests too inside
Eclipse using Yourkit. Anyway I came to a final conclusion:
*Conclusion*: We need some repeatable benchmarking/profiling perfromance
test suite for ApacheDS that can be run easily.
Requirements for BPPS
Here's what I started asking myself for internally. Please add to this
list if you can think of other requirements.
(1a) Need repeatable performance tests with setup prep and tear down
(1b) Tests should be able to load an initial data set (LDIF) into server
(2) I should be able to use Maven or ant to kick off these tests
(3) Tests should produce some kind of report
(4) Tests should easily be pointed to benchmark other servers
(5) Make it easy to create a new performance test.
(6) I want a summary of the conditions in the test report which include
the setup parameters for:
o operations performed
o operating system
We also need differnet kind of tests. Many parts of the server can be tested so far, so micro-benchmarks may be added. A sub-project is needed for this
Existing work and potential approaches
I figured using JUnit was the best way to test ApacheDS or anyother
server. Plus I could setUp and tearDown test cases. The only thing I
needed to do was make a base test case or two for the various apacheds
configurations (embedded testing verses full networked testing).
The first base test case, for embedded testing, was setup here:
Yeah it's weak and I'll try to add to it. What I would like to do is
invite people to work with me on setting up this
benchmarking/profiling/perf testing framework.