directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Quanah Gibson-Mount <qua...@stanford.edu>
Subject Re: [ApacheDS] Performance testing
Date Wed, 28 Jun 2006 18:06:48 GMT


--On Wednesday, June 28, 2006 12:23 PM -0400 Alex Karasulu 
<aok123@bellsouth.net> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Need Benchmarking/Profiling Perf Suite (BPPS) for ApacheDS
> ==========================================================
>
> I just sat down in front of my machine and wanted to run some
> performance tests on the server.  These questions came up:
>
> o What tests do I run?
> o What LDIF should I import first before running those tests?
> o Which version of the server should I run these tests against?
> o Can I run these tests against other LDAP servers?
>
> I did the first thing anyone would do.  I tapped Emmanuel on the
> shoulder to ask him for his materials for his AC EU presentation.  I did
> not want to repeat the work that he had already done.
>
> Please Emmanuel take no offense but I found the setup and repeated work
> to be a bit of a hassle.  I'm sure you were bothered by doing things
> manually yourself.  Plus I wanted to profile these tests too inside
> Eclipse using Yourkit.  Anyway I came to a final conclusion:
>
> *Conclusion*: We need some repeatable benchmarking/profiling perfromance
> test suite for ApacheDS that can be run easily.
>
>
> Requirements for BPPS
> =====================
>
> Here's what I started asking myself for internally.  Please add to this
> list if you can think of other requirements.
>
> (1a) Need repeatable performance tests with setup prep and tear down
> (1b) Tests should be able to load an initial data set (LDIF) into server
> (2) I should be able to use Maven or ant to kick off these tests
> (3) Tests should produce some kind of report
> (4) Tests should easily be pointed to benchmark other servers
> (5) Make it easy to create a new performance test.
> (6) I want a summary of the conditions in the test report which include
> the setup parameters for:
> 	o operations performed
> 	o capacity,
>  	o concurrency,
> 	o hardware.
> 	o operating system
>
>
> Existing work and potential approaches
> ======================================
>
> I figured using JUnit was the best way to test ApacheDS or anyother
> server.  Plus I could setUp and tearDown test cases.  The only thing I
> needed to do was make a base test case or two for the various apacheds
> configurations (embedded testing verses full networked testing).
>
> The first base test case, for embedded testing, was setup here:
>
> http://svn.apache.org/viewvc/directory/trunks/apacheds/core-unit/src/main
> /java/org/apache/directory/server/core/unit/AbstractPerformanceTest.java?
> revision=414035&view=markup
>
> Yeah it's weak and I'll try to add to it.  What I would like to do is
> invite people to work with me on setting up this
> benchmarking/profiling/perf testing framework.
>
> Comments? Thoughts?


I'd note that SLAMD already does a lot of what you are discussing, is open 
source, and is written in Java.  What it doesn't do is load the LDIF into 
the server.  However, these are the tools that it does have:

(a) Ability to use very sophisticated templates to generate LDIF files for 
loading into directory servers.
(b) Distributed clients -- You generally cannot get an idea of the 
performance of an LDAP server from a single client
(c) Resource monitoring -- This monitors CPU usage, Disk usage, SWAP, and 
Memory, maybe other things as well
(d) A number of pre-defined benchmarking tests as well as the ability to 
create your own
(e) Report generation, including PDF and HTML


--Quanah

--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html

Mime
View raw message