Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (asf.osuosl.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
  b=IyJtsc+Kz47t+2WXxF3xqr3fQwuzMik7qExGzpfZm+UHwvuLL8I0mXerys6EW5KK3Rq0Dx25SfN+MRb330SqmHz36f3Nrb6CLj37f1AmcVxLRfMnTtpieX3mpXSbdkOe0UhmlDWl/jcgpfUb99h809J6fWaIoHlkesW4aHBCGYI=
  ;
Message-ID: <20060628172349.64751.qmail@web60723.mail.yahoo.com>
Date: Wed, 28 Jun 2006 10:23:49 -0700 (PDT)
From: Ole Ersoy <ole_ersoy@yahoo.com>
Subject: Re: [ApacheDS] Performance testing
To: Apache Directory Developers List <dev@directory.apache.org>
In-Reply-To: <44A2AD0C.60300@bellsouth.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 8bit

Hey Alex,

Welcome back from vacation!

Here's a couple of thoughts:

- Created maven archetypes specific to performance
testing directory servers, with a set of directories
for DS configuration files.  Thus there should be one
archetype per DS.  So apache would have one, and
OpenLDAP would have another.  The configuration files
directory should mirror the DS's configuration file
directory structure.  This isolates the DS's
configuration settings per performance test, and
provides the initial baseline for everyone to use.
- Create another Archetype input directory for the
LDIF files. 
- Write a mojo that embeds ApacheDS, loads the
configuration files, and then runs all the tests on
the specific server instance, and then generates
reports.  This way each instance of this archetype is
running the DS in an isolated manner.  Would also be
cool to integrate this stuff with Eclipse TPTP, using
something similar to the maven eclipse plugin...that
way all the results are cross compatible with TPTP and
can be opened up in an eclipse TPTP project...maybe in
version 2 of the mojo...or mojo plugin set.
- Something similar for OpenLDAP...
- Bind this mojo to the maven lifecycle in each
archetype.
- Now anyone can create a quick maven DS performance
testing project, make a few configuration file changes
if that's what's being tested, add unit tests if
necessary, and go!
- The mojo should also produce put the reports in an
output directory somewhere within the site directory,
and this directory should be the same for all DSs.

I'll be glad to throw together a sample archetype for
ApacheDS if this sounds interesting.

Coding the mojo should also be fairly straightforward.

Thoughts?

Cheers,
- Ole


- Write a maven mojo that 

--- Alex Karasulu <aok123@bellsouth.net> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Need Benchmarking/Profiling Perf Suite (BPPS) for
> ApacheDS
>
==========================================================
> 
> I just sat down in front of my machine and wanted to
> run some
> performance tests on the server.  These questions
> came up:
> 
> o What tests do I run?
> o What LDIF should I import first before running
> those tests?
> o Which version of the server should I run these
> tests against?
> o Can I run these tests against other LDAP servers?
> 
> I did the first thing anyone would do.  I tapped
> Emmanuel on the
> shoulder to ask him for his materials for his AC EU
> presentation.  I did
> not want to repeat the work that he had already
> done.
> 
> Please Emmanuel take no offense but I found the
> setup and repeated work
> to be a bit of a hassle.  I'm sure you were bothered
> by doing things
> manually yourself.  Plus I wanted to profile these
> tests too inside
> Eclipse using Yourkit.  Anyway I came to a final
> conclusion:
> 
> *Conclusion*: We need some repeatable
> benchmarking/profiling perfromance
> test suite for ApacheDS that can be run easily.
> 
> 
> Requirements for BPPS
> =====================
> 
> Here's what I started asking myself for internally. 
> Please add to this
> list if you can think of other requirements.
> 
> (1a) Need repeatable performance tests with setup
> prep and tear down
> (1b) Tests should be able to load an initial data
> set (LDIF) into server
> (2) I should be able to use Maven or ant to kick off
> these tests
> (3) Tests should produce some kind of report
> (4) Tests should easily be pointed to benchmark
> other servers
> (5) Make it easy to create a new performance test.
> (6) I want a summary of the conditions in the test
> report which include
> the setup parameters for:
> 	o operations performed
> 	o capacity,
>  	o concurrency,
> 	o hardware.
> 	o operating system
> 
> 
> Existing work and potential approaches
> ======================================
> 
> I figured using JUnit was the best way to test
> ApacheDS or anyother
> server.  Plus I could setUp and tearDown test cases.
>  The only thing I
> needed to do was make a base test case or two for
> the various apacheds
> configurations (embedded testing verses full
> networked testing).
> 
> The first base test case, for embedded testing, was
> setup here:
> 
>
http://svn.apache.org/viewvc/directory/trunks/apacheds/core-unit/src/main/java/org/apache/directory/server/core/unit/AbstractPerformanceTest.java?revision=414035&view=markup
> 
> Yeah it's weak and I'll try to add to it.  What I
> would like to do is
> invite people to work with me on setting up this
> benchmarking/profiling/perf testing framework.
> 
> Comments? Thoughts?
> 
> Thanks,
> Alex
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird -
> http://enigmail.mozdev.org
> 
>
iD8DBQFEoq0LPVto+tI7JJARAuwJAJ4rCuEUoxPRPMVrPOGyWmymqQvcFwCdEl6U
> 6h7gC9odQUjn0iyXTg/8gPs=
> =UUJD
> -----END PGP SIGNATURE-----
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com