db-ojb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Giordano <giord...@more.net>
Subject Throughput testing OJB with Apache Benchmark (AB)
Date Fri, 05 Dec 2003 16:28:48 GMT
OJB Developers:

Back in the middle of October (10/15) we sent feedback to this list 
regarding our experience with using OJB (OJB usability parts 1-5).  With 
one issue, we expressed our serious concerns regarding the performance 
of OJB that primarily caused us to back out of using the technology.

In response to the discussion that followed our postings we wanted to 
validate our observations regarding OJB performance.  In particular, we 
made a false assumption regarding the operation of the cache.  We were 
observing at least a 5-fold decrease in performance using OJB and this 
lead us an erroneous conclusion there were significant design flaws in 
the OJB api that were severely affecting performance.

We used Apache Benchmark to do end-to-end throughput testing of a simple 
servlet running within Tomcat (v4.1.27) that performs queries to a 
MySQLdatabase to retrieve a single object.  It was derived largely from 
examples that are part of the OJB documention available on the projects 
website.  We used OJB v1.0 rc4 installed as part of the servlet's Web 
Application context (as opposed to globally within the servlet 
container).  We changed one property in the OJB.properties file.  We 
configured the the following ConnectionFactoryClass:


I've attached the full output of the Apache Benchmark results of four 
tests each run with three levels of thread concurrency loading.  In 
summary, these benchmarking results largely coincide with the 
performance documentation on the OJB website 

Of note was the test using OQL w/25 threads.  This test caused Java 
Exceptions to be thrown.  Cranking up the number of concurrent threads 
using AB allowed this problem to manifest itself more quickly.  Lower 
values also caused the Exceptions but only after several successful 
iterations of testing.  At this point, we do not know whether this 
problem lies within the servlet design, a problem with our local 
system's configuration, an OJB configuration or the OJB api itself.

Our original observations that demonstrated a 5-fold decrease in 
performance can still be observed (results of these tests have not been 
submitted). But, we have been able to localize the problem to using OJB 
profiles to store per-thread, user defined class mappings that are 
loaded dynamically at runtime. Therefore, it appears we have found a way 
to use and/or configure OJB that results in unfavorable performance 
gains.  But this cannot necessarily be attributed to the OJB api as a 
whole.  We hope to do a further investigation sometime in the future and 
arrive at the root cause of the problem.  At that time, we'll pass along 
any interesting results to the list.

In conclusion, our testing results, in fact, appear favorable toward 
OJB.  We would, therefore,  like to recognize our undue criticism 
regarding OJB performance issues that were grounded in false 
assumptions.  Our intentions were not to introduce confusion.  The 
responses we received from our original postings to the list were most 
always helpful and productive.

Chris Giordano

View raw message