directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Karasulu <>
Subject Re: [ApacheDS] Performance testing
Date Thu, 29 Jun 2006 20:04:50 GMT
Hash: SHA1

Ole Ersoy wrote:
> Hey Alex,

Hey Ole good to hear from you again.

> Welcome back from vacation!

Thanks.  It was too short.

> - Created maven archetypes specific to performance
> testing directory servers, with a set of directories
> for DS configuration files.  

Hmmm I see where you're going.  You want to generate projects that have
 a server.xml that can be customized for the test case.  What about
having test cases for embedded operation where the server.xml does not
come into play?

Thus there should be one
> archetype per DS.  So apache would have one, and
> OpenLDAP would have another.  

Ok I see now.  The OpenLDAP archetype is a bit more difficult to pull
off tho since it is C-code and not embeddable or launchable via test
cases.  We might have to fork an instance of it.

The configuration files
> directory should mirror the DS's configuration file
> directory structure.  This isolates the DS's
> configuration settings per performance test, and
> provides the initial baseline for everyone to use.

So by using an archetype you plan to isolate each performance test case
into a separate maven module?

> - Create another Archetype input directory for the
> LDIF files. 

Ok starting to loose you here.  Perhaps my knowledge of maven archetypes
are breaking down now.  Can you elaborate?

> - Write a mojo that embeds ApacheDS, loads the
> configuration files, and then runs all the tests on
> the specific server instance, and then generates
> reports.  

Yeah this I understand.  We can do this pretty easily.

This way each instance of this archetype is
> running the DS in an isolated manner.  

Hmmm again loosing you here on the meaning or use of the m2 archetype.

Would also be
> cool to integrate this stuff with Eclipse TPTP, using
> something similar to the maven eclipse plugin...that
> way all the results are cross compatible with TPTP and
> can be opened up in an eclipse TPTP project...maybe in
> version 2 of the mojo...or mojo plugin set.

Is this the Eclipse profiling stuff?  This integration would be pretty
tight.  Perhaps as you suggest a bit overkill for a first round.

What I'm looking to do is to work with a couple guys here over the
course of the next 1-3 weeks to erect a solid framework.  If we can add
some of this profiler spice it would be really cool but not a
requirement.  FYI I'm a big fan of Yourkit but free tools are much more

> - Something similar for OpenLDAP...
> - Bind this mojo to the maven lifecycle in each
> archetype.
> - Now anyone can create a quick maven DS performance
> testing project, make a few configuration file changes
> if that's what's being tested, add unit tests if
> necessary, and go!

Sounds really fun to do: it's a nice little project and the utility will
be high.  I like the idea of having different mojos for each LDAP server
that needs to be setup and leaving setup tasks to the mojo/archetype

> - The mojo should also produce put the reports in an
> output directory somewhere within the site directory,
> and this directory should be the same for all DSs.

This will be nice to have ... an easily published set of results.

> I'll be glad to throw together a sample archetype for
> ApacheDS if this sounds interesting.

I'm interested in the idea.  I'd love to take a look at it.  My Maven 2
archetype skills are non-existent tho.  I need to brush them up.

> Coding the mojo should also be fairly straightforward.

Yeah I agree plugins are a cake walk in m2.


 o [Observation] As Quanah suggested in this thread, we need more than
one client to really stress test an LDAP server for total throughput
under high concurrency.  This however is one kind of test we will
perform: there are also capacity tests as well.  We can still use the
recommended tools (SLAMD or another), the partial goal of this effort is
to reduce the setup, and running of LDAP servers to be tested in various
ways.  We want to automate the setup process with an easy way to
create/setup a harness for various LDAP servers with similar testing

 o Why Maven?  It is important to figure out why we (me too btw) want to
use Maven.  OK PLEASE DON'T FLAME ME ON THIS GUYS :D.  First off it's
our build tool of choice for this project.    Secondly it can run test
cases, and fork them if need be with new JVM settings.  It's got a nice
plugin architecture and has these things for creating project templates
called archetypes.

BTW Ole, you caught me by surprise when you suggested using archetypes
and plugins (mojos) for this effort.  The idea gives me a good feeling
even though I don't know why yet ... there must be many good advantages
to this.


> - Write a maven mojo that 
> --- Alex Karasulu <> wrote:
> Need Benchmarking/Profiling Perf Suite (BPPS) for
> ApacheDS
>> ==========================================================
> I just sat down in front of my machine and wanted to
> run some
> performance tests on the server.  These questions
> came up:
> o What tests do I run?
> o What LDIF should I import first before running
> those tests?
> o Which version of the server should I run these
> tests against?
> o Can I run these tests against other LDAP servers?
> I did the first thing anyone would do.  I tapped
> Emmanuel on the
> shoulder to ask him for his materials for his AC EU
> presentation.  I did
> not want to repeat the work that he had already
> done.
> Please Emmanuel take no offense but I found the
> setup and repeated work
> to be a bit of a hassle.  I'm sure you were bothered
> by doing things
> manually yourself.  Plus I wanted to profile these
> tests too inside
> Eclipse using Yourkit.  Anyway I came to a final
> conclusion:
> *Conclusion*: We need some repeatable
> benchmarking/profiling perfromance
> test suite for ApacheDS that can be run easily.
> Requirements for BPPS
> =====================
> Here's what I started asking myself for internally. 
> Please add to this
> list if you can think of other requirements.
> (1a) Need repeatable performance tests with setup
> prep and tear down
> (1b) Tests should be able to load an initial data
> set (LDIF) into server
> (2) I should be able to use Maven or ant to kick off
> these tests
> (3) Tests should produce some kind of report
> (4) Tests should easily be pointed to benchmark
> other servers
> (5) Make it easy to create a new performance test.
> (6) I want a summary of the conditions in the test
> report which include
> the setup parameters for:
> 	o operations performed
> 	o capacity,
>  	o concurrency,
> 	o hardware.
> 	o operating system
> Existing work and potential approaches
> ======================================
> I figured using JUnit was the best way to test
> ApacheDS or anyother
> server.  Plus I could setUp and tearDown test cases.
>  The only thing I
> needed to do was make a base test case or two for
> the various apacheds
> configurations (embedded testing verses full
> networked testing).
> The first base test case, for embedded testing, was
> setup here:
> Yeah it's weak and I'll try to add to it.  What I
> would like to do is
> invite people to work with me on setting up this
> benchmarking/profiling/perf testing framework.
> Comments? Thoughts?
> Thanks,
> Alex

> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around 

Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla -


View raw message