commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Craig R. McClanahan" <craig...@apache.org>
Subject [BeanUtils] Interesting Microbenchmarks
Date Sat, 02 Aug 2003 21:37:06 GMT
TheServerSide (www.theserverside.com) recently published a case study that
attempted to analyze the performance characteristics of a variety of
middleware configurations, including two J2EE app servers and Microsoft's
.NET platform.  One of the hot spots that showed up in the Java based
figures was identified as heavy use of reflection (the test app was Struts
based and therefore uses commons-beanutils), but the report did not detail
these findings.

Despite the fact that the test condtions were very artificial (the
database was tuned in a way that it was never the hotspot, which is unlike
virtually any high-use web application in production), it prompted a bit
of discussion over whether using reflection was a good idea.  This in turn
prompted me to write a couple of simple microbenchmarks to measure the
behavior of the most commonly used BeanUtils and PropertyUtils methods for
copying sets of properties -- copyProperties() and populate().

Legend:
======

  "bean" - A plain old Java object (POJO) with eight properties,
           one of each of the standard primitive types and one
           String (org.apache.commons.beanutils.BenchBean)

  "dyna" - An instance of org.apache.commons.beanutils.BasicDynaBean
           with exactly the same properties and types.  The property
           types for primitives in a DynaBean are the wrapper classes
           such as java.lang.Integer.

  "map"  - An instance of HashMap with eight entries pointing at
           exactly the same values.  The values for primitives
           of the corresponding wrapper class types.

  "strs" - An instance of HashMap with eight entries pointing at
           the same values, but all Strings (simulating request
           parameters that are used to populate a form bean in Struts).

  "Dura #1" - Duration for 100,000 loops on my 1ghz desktop (milliseconds)

  "Dura #2" - Duration for 50,000 loops on my 500mhz laptop (milliseconds)

I tested all the legal combinations of an origin object (bean, dyna,
map, or strs) to a destination object (bean or dyna).  The tests were run
on a 1 gigahertz PC running Linux (Red Hat 9) with Sun's JDK 1.4.2, and on
a 500mhz laptop running Win2K with Sun's 1.4.2.  The loop count was set to
100,000 in the PC case, and 50,000 in the laptop case, so the overall
execution times should be roughly the same (since the desktop is roughly
twice as fast).  Execution time was measured in milliseconds.  For each
test, I ran an untimed execution of the specified number of iterations (to
prime HotSpot), followed by a timed execution of the same number of
iterations.

To run these for yourselves, check out the HEAD branch of beanutils (or
grab a source nightly distro dated 20030803 or later), and run "ant
bench".

Results for BeanUtils (which performs type conversions):
=======================================================

Method         Dest Orig Dura #1 Dura #2
-------------- ---- ---- ------- -------

copyProperties bean bean  17,703  17,075
copyProperties dyna bean  17,463  17,074
copyProperties bean dyna  15,413  15,042
copyProperties dyna dyna  13,020  12,077
copyProperties bean map   15,626  15,192
copyProperties dyna map   12,999  12,237
copyProperties bean strs  16,302  16,093
copyProperties dyna strs  13,940  13,129

populate       bean map    7,438   7,481
populate       dyna map    5,082   5,368
populate       bean strs   5,768   5,739
populate       dyna strs   3,865   3,746


Results for PropertyUtils (no type conversions):
===============================================

Method         Dest Orig Dura #1 Dura #2
-------------- ---- ---- ------- -------

copyProperties bean bean   5,595   5,107
copyProperties dyna bean   4,567   4,126
copyProperties bean dyna   3,791   3,675
copyProperties dyna dyna     844     681
copyProperties bean map    3,938   3,755
copyProperties dyna map      931     772


Observations:
============

* The absoute numbers are not particularly interesting, other than the
  fact that they confirm my belief that copying properties (whether
  using reflection or not) is very unlikely to be a bottleneck -- even
  on bargain basement hardware (in today's world), we're talking about
  copying all the properties in anything from 0.8 microseconds up to
  177 microseconds.

* This is a very limited test case, with only one pattern of properties.
  I suspect it will be on the worst case side in terms of the number
  of conversions required, compared to how most people use these methods
  in real applications.

* Increasing the number of properties being copied will (of course)
  increase the execution times, but I suspect it will not change the
  ratios between alternatives much.  I've done a few informal tests
  with more and fewer properties, but this will be worth investigating
  in more detail as well.

* Only the cases where a "bean" was the destination or origin use
  reflection at all.  The "map" case is just doing HashMap gets and puts,
  while the "dyna" case is really the "map" case plus type safety
  checks.

* In all scenarios, using "dyna" was faster than using "bean", ranging
  from statistically insignificant to pretty substantial percentages.
  For Struts based apps in particular, that means using DynaActionForm
  for form beans, with the current BeanUtils code, will give better
  performance than using standard ActionForm bean classes.  It also
  seems that BasicDynaBean would make a pretty good Data Transfer Object
  for moving data between tiers, adding type safety without giving up
  the performance of using just a HashMap.

* One would expect using a "map" as the origin would be roughly
  the same as using a "dyna", because they both iterate over keys and
  values in a HashMap internally.  The fact that using "dyna" was often
  faster is somewhat anomalous, and is likely based on differing amounts
  of optimization inside the BeanUtils and PropertyUtils loops and
  getter/setter implementations.

* Of particular interest to Struts users is the code that populates
  the form bean.  The two lines that describe this use case (populate
  to "bean" from "strs", and populate to "dyna" from "strs") indicate
  that this operation is already highly optimized, running much
  faster than the more general purpose copyProperties() method with the
  same argument types.

* From casual observation of the current source, I suspect the longer
  times for "bean" objects have more to do with the way that
  PropertyDescriptor and Method instances are cached and searched in
  PropertyUtils than they have with the time of the reflection-based
  execution of the property getters and setters.  There is lots of room
  here to optimize.

* It will be very useful to have people try these microbenchmarks in a
  variety of different environments, and create variations of them with
  different kinds of origin and destination beans, to provide a wider
  set of profiles when we attempt optimization on the underlying code.
  Please feel free to post patches to the microbenchmark sources, so that
  everyone can execute exactly the same set of tests.

* It will also be interesting to try this on different JDK generations,
  to confirm whether the improvements in reflection performance that have
  been described for recent versions shows through in the results here.

Craig McClanahan


Mime
View raw message