Return-Path: Delivered-To: apmail-incubator-geronimo-dev-archive@incubator.apache.org Received: (qmail 27558 invoked by uid 500); 9 Aug 2003 13:39:53 -0000 Mailing-List: contact geronimo-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: list-post: Reply-To: geronimo-dev@incubator.apache.org Delivered-To: mailing list geronimo-dev@incubator.apache.org Received: (qmail 27540 invoked from network); 9 Aug 2003 13:39:53 -0000 Received: from h013.c001.snv.cp.net (HELO c001.snv.cp.net) (209.228.32.127) by daedalus.apache.org with SMTP; 9 Aug 2003 13:39:53 -0000 Received: (cpmta 24230 invoked from network); 9 Aug 2003 06:39:54 -0700 Received: from 68.107.221.139 (HELO downinthedesert.com) by smtp.register-admin.com (209.228.32.127) with SMTP; 9 Aug 2003 06:39:54 -0700 X-Sent: 9 Aug 2003 13:39:54 GMT Message-ID: <3F34F9AA.9050203@downinthedesert.com> Date: Sat, 09 Aug 2003 06:39:54 -0700 From: "John C. Dale" Reply-To: jcd@downinthedesert.com Organization: Down in the Desert, Inc. User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20020830 X-Accept-Language: en-us, en MIME-Version: 1.0 To: geronimo-dev@incubator.apache.org Subject: Re: Reflection Bad, OO and direct Method invocation Good... References: Content-Type: multipart/alternative; boundary="------------090404040005060309040505" X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N --------------090404040005060309040505 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Microbenchmarks can in fact be misleading when we aren't talking about as much as a factor of 10! Reflection, in the best case scenario below, was a factor of 10 slower! The bottom line is that reflection takes a lot more electricity than does non-reflection. You are absolutely right in that the round trip to the database is going to consume more time for the life of the user request than is a sequence of reflection-based calls. That will always be the bottleneck in systems that don't have any kind of intelligent middle-tier cache - but wait, there's more! Entity EJBS are just that (a cache) aren't they? Through intelligent caching and good design of our applications, we're able to avoid the extra knocks on the database if the entity of our type with our PK has already been loaded. This, in my mind, is one of the chief advantages of using Entity Beans; throughput. This throughput will definitely be impacted negatively if there are several reflection calls for each user request as the system comes under (and maintains) load. Furthermore, although the bottleneck is clearly the round trip to the database for the hydration/storage of data, the time it takes to get to the point of JDBC query invocation will be effected negatively if reflection is used - this effect becomes more pronounced as the system comes under heavy load. The sooner the JDBC query starts, the sooner it will finish. I would also note that 'optimizing away' performance issues due to reflection (assuming that the core of the system is based on reflection) is MUCH easier to type than it is to implement. If we see the train coming now, why don't we get out of the way...now? Best, John C. Dale Professional Services Compuware Aaron Mulder wrote: > First of all, can we agree that microbenchmarks are dangerous? > > But all that aside, try 1 million HTTP requests, or 1 million >iterations of a JDBC query, and then provide a ratio between JDBC/HTTP >time and reflection time. If you can perform a million requests of either >type in 30 seconds I'd be shocked, and that's two orders of magnitude. > > IMHO, reflection is something that can be optimized away later, >once it becomes the bottleneck on performance. I don't think we need to >avoid it from the outset, because I don't think it will be the bottleneck >at the outset. > >Aaron > >On Sat, 9 Aug 2003, John C. Dale wrote: > > >>I'm concerned about your mention of reflection. I really like >>reflection, and think it has many useful applications in the application >>development space, but should be kept out of any runtime application >>that has stringent scalability and performance requirements. Although >>reflection allows an incredibly generic way to solve many difficult >>problems, and really opens the door to completely grandios perfectly >>idealistic genericititousness ;) , a straight-forward OO approach to >>solving the problems would 1) be faster and 2) be easier to read and >>maintain. I've seen reflection based approaches work, and I've seen >>them bomb. >> >>One might argue that in a container based system that will eventually >>support clustering and scalability doesn't have to worry about savoring >>individual CPU cycles that might be expended needlessly with a >>reflection-based design. Just add more hardware and the user experience >>will be perpetually preserved. I would argue, however, that the >>development community will be able to make better use of the product if >>the rate at which additional hardware purchases becomes necessary >>decreases. The philosophy, IMHO, should be to solve the problems at >>hand with straight-forward OO solutions, and not to focus so much on >>being generic and supporting n * infinity solutions up-front. Using >>things like the Policy/Strategy pattern at well-selected locations will >>afford the opportunity for pluggability without compromising >>maintainability and performance. >> >>Here are some dramatic results: >> >>*Direct:* >> >>public void init() >>{ >> customer = new Customer(); >>} >> >>public void run() >>{ >> customer.work(); >>} >> >> >>*Interface (Polymorphism):* >> >>public void init() >>{ >> customer = new Customer(); >>} >>public void run() >>{ >> Person person = (Person) customer; >> person.work(); >>} >> >> >>*Reflection:* >> >>public void init() >>{ >> customer = new Customer(); >> try >> { >> method = customer.getClass().getMethod("work", new Class[0]); >> } >> catch (Exception e) >> { >> } >> >> params = new Object[0]; >>} >> >>public void run() >>{ >> try >> { >> method.invoke(customer, params); >> } >> catch (Exception e) >> { >> } >>} >> >> >>With 1000000 of the above code, here were the results: >> >>JDK DirectTest InterfaceTest ReflectionTest >>*Sun 1.4* 52 ms 54 ms 543 ms >>*Sun 1.4 -server* 26 ms 56 ms 279 ms >>*Sun 1.3* 124 ms 128 ms 2168 ms >>*Sun 1.3 -server* 41 ms 58 ms 2012 ms >>*IBM 1.3* 75 ms 78 ms 2134 ms >> >> >>Reflection will significantly effect the performance of the system and >>should be avoided for any runtime operations. Where it should and could >>definitely be applied is with the dynamic generation and compilation of >>code, or generation of metadata. Otherwise, even at the recommendation >>of Sun, it should be avoided for runtime operations. >> >>Best, >> >>John C. Dale >> >>Leo Simons wrote: >> >> >> >>>Jason Dillon wrote: >>> >>> >>> >>>>PS. Can someone write up something about the current state of the >>>>major component containers out there with a feature blurb... no soap >>>>boxes, just the facts jack. >>>> >>>> >>>What, no soap boxes? How on earth can anyone comply with that? ;) No >>>wait, >>>you weren't asking me anyway, were you. Ah, e-mail already typed. Bummer. >>> >>>= My Opinion = >>> >>>You should *not* be evaluating component containers. You should save >>>this discussion >>>for a later date and just code your way to 1.0. The basic design idea >>>overview >>>below should show that most of the architectural concepts behind all >>>these containers are >>>very similar. I've been experimenting (no, I will not plug it, you >>>will just get confused) with >>>some reflection that will allow any component written for any of the >>>below to run in any >>>of the other containers, and that is feasible, straightforward and >>>performant. >>> >>>So write your components to plug in whatever you have, do a nice IoC, >>>SoC, SAI, AOP >>>setup, and you will be able to defer refactoring around an external >>>container until much later. >>> >>>But that's my opinion, and I have now said it three times, and your a >>>responsible adult >>>(yep, its a guess, you could also be 11 years old :D). Switching >>>soapbox mode off. >>> >>> >>>= Disclaimer = >>> >>>Comparing component containers is comparing apples with pears. Avalon >>>is by far the >>>biggest 'generic' project at the moment, for example, but recent >>>developments utilize AOP >>>and interceptor architecture to support a much 'lighter' >>>container-component api and >>>contract. Indeed, picocontainer was started by an avalon elder from >>>the firm belief that >>>things should be simpler and smaller. So to actually evaluate all this >>>stuff, you really should >>>spend a day or so delving into the websites and the code of all these >>>projects, and backing >>>tech like nanning (nanning.codehaus.org) and aspectj >>>(www.aspectj.org). I would start >>>by looking at pico/nano and xwork, then take a look at the tutorials >>>for avalon-merlin. >>>The other projects have a smaller community atm, and I tend to value >>>community size >>>and vibe. >>> >>>Furthermore, I have strong opinions about stuff, and allegations to >>>various projects, hence >>>this is not an objective overview, even though I tried to make it >>>somewhat objective. >>> >>> >>>= Features/ design idea shorthands = >>> >>>IoC = Inversion of Control, the idea that an application is controlled >>>from the top down >>>SoC = Seperation of Coccerns, the idea that a class (aspect) should do >>>one job and do it well >>>SAI = Seperation of API from Implementation, the idea that you define >>>and code to work >>> interfaces >>>AOP = Aspect Oriented Programming, mostly lightweight nowadays where >>>you add a chain >>> of interceptors around a method call that can handle orthogonal >>>concerns >>>DecP = Declarative Programming, where you use a declarative-style >>>language (usually xml) to >>> determine things like component wiring (ie your average tomcat >>>config file, generalized) >>>EBP = Event Based Programming, basically making the inter-object >>>method call asynchronous >>> and encapsulating such a call into some kind of event object that >>>can be queued, modfied, >>> etc >>> >>> >>>= No particular order, incomplete list = >>> >>>http://wiki.opensymphony.com/space/XWork - IoC, SoC, SAI, AOP, DecP, >>>EBP. Nearing >>>1.0 release. Used in webwork2 (a competitor to struts). EBP very basic >>>only. Lean and mean, >>>but not mature and some client-server web-layer specific assumptions. >>>Don't like the XXXAware >>>interfaces. Very vibrant and active community and many famous peeps >>>with J2EE experience >>>around at opensymphony. >>> >>>http://www.picocontainer.org/ and http://www.nanocontainer.org/ - >>>IoC, SoC, SAI, AOP, DecP. 1.0 beta releases. Lean and mean and very >>>extensible and >>>embeddable, developed by smart XP peeps, some stuff already in use in >>>some apps, but >>>otherwise pretty much alpha. I love picocontainer and the way they're >>>doing the project. >>>The dev community is intentionally kept small atm, but many peeps are >>>watching this one. >>> >>>http://plexus.codehaus.org/ - IoC, SoC, SAI, DecP. Container supporting >>>avalon-framework components. Used to be >>>yet-another-novel-avalon-container, but I >>>think they're growing to be container-component-contract-agnostic. >>>Corporate backed >>>development. Smart guys, not so much focussed on releases as on >>>getting all the >>>functionality they need (which is a lot) in place. Very much a >>>pragmatic project. >>> >>>http://www.jcontainer.org/ - Not yet public container development >>>(dubbed 'loom' IIRC) by a smart ex-avaloner 'n others. Haven't seen >>>any code yet >>>but my guess is it'll be good. The website says "move along" so you >>>prolly should. >>> >>>http://www.springframework.org/ - haven't looked at in too much depth. >>>Seems similar to >>>nanocontainer and xwork. Think it has one active developer and a beta >>>release. Some >>>smart points made, but too much xml for my taste. Hoping to see some >>>of this rolled into >>>Xwork and/or pico. >>> >>>http://avalon.apache.org/ - IoC, SoC, SAI, DecP, EBP (EBP for fortress >>>only). By far >>>the oldest 'generic container' project. Big committer base, mature >>>codebase, mature >>>ASF project (which can be a good thing and a bad thing ;) rather >>>extensive >>>'avalon-framework' (comparatively heavy compared to more recent >>>developments) >>>that defines the contracts between a component and a container. Has 3 >>>container projects to consider: avalon-phoenix, a mature microkernel >>>design, >>>avalon-fortress, similar in weight and featureset to something like >>>nanocontainer with a >>>1.0 release (successor to avalon-ecm, the container used in (among >>>other projects) >>>apache-cocoon), and avalon-merlin, a more recent development which we >>>_seem_ >>>to be converging on as the successor to all other previously produced >>>containers. Merlin >>>is further described in the email by Stephen Mcconnell. Arguably the >>>biggest, >>>most extensive and most dynamic IoC container implementation around >>>(and hence >>>also the most complex). Re: my earlier blurb on 'geronimo and avalon' >>>for more >>>yadayada. >>> >>>http://jakarta.apache.org/commons/sandbox/hivemind - haven't looked at >>>in much detail, but seems >>>very similar in scope and setup to avalon at first glance. Framework >>>being refactored out >>>of hibernate, one developer, still in alpha with no releases I think. >>>No offense to Howard >>>intended, but I think he's cut himself a rather big piece of the >>>puzzle to recode from scratch >>>at once. But I am an uninformed whiner, so I'm not going to comment >>>further in the hope that >>>Howard will just eventually see the light and direct his energy >>>towards collaboration with the >>>avalon peeps :P >>> >>> >>>I am spending too much time on writing messages to this mailing list >>>(it is nearly 3am over here). >>>I promise this is the last message from me for two weeks :D >>> >>>g'night, >>> >>>- Leo >>> >>> >>> >>> >>> >>> >>> >> >> > > > > > --------------090404040005060309040505 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Microbenchmarks can in fact be misleading when we aren't talking about as much as a factor of 10!  Reflection, in the best case scenario below, was a factor of 10 slower!  The bottom line is that reflection takes a lot more electricity than does non-reflection.

You are absolutely right in that the round trip to the database is going to consume more time for the life of the user request than is a sequence of reflection-based calls.  That will always be the bottleneck in systems that don't have any kind of intelligent middle-tier cache - but wait, there's more!

Entity EJBS are just that (a cache) aren't they?  Through intelligent caching and good design of our applications, we're able to avoid the extra knocks on the database if the entity of our type with our PK has already been loaded.  This, in my mind, is one of the chief advantages of using Entity Beans; throughput.  This throughput will definitely be impacted negatively if there are several reflection calls for each user request as the system comes under (and maintains) load.  Furthermore, although the bottleneck is clearly the round trip to the database for the hydration/storage of data, the time it takes to get to the point of JDBC query invocation will be effected negatively if reflection is used - this effect becomes more pronounced as the system comes under heavy load.  The sooner the JDBC query starts, the sooner it will finish.

I would also note that 'optimizing away' performance issues due to reflection (assuming that the core of the system is based on reflection) is MUCH easier to type than it is to implement.  If we see the train coming now, why don't we get out of the way...now?

Best,

John C. Dale
Professional Services
Compuware


Aaron Mulder wrote:
	First of all, can we agree that microbenchmarks are dangerous?

	But all that aside, try 1 million HTTP requests, or 1 million
iterations of a JDBC query, and then provide a ratio between JDBC/HTTP
time and reflection time.  If you can perform a million requests of either
type in 30 seconds I'd be shocked, and that's two orders of magnitude.

	IMHO, reflection is something that can be optimized away later,
once it becomes the bottleneck on performance.  I don't think we need to 
avoid it from the outset, because I don't think it will be the bottleneck 
at the outset.

Aaron

On Sat, 9 Aug 2003, John C. Dale wrote:
  
I'm concerned about your mention of reflection.  I really like 
reflection, and think it has many useful applications in the application 
development space, but should be kept out of any runtime application 
that has stringent scalability and performance requirements.  Although 
reflection allows an incredibly generic way to solve many difficult 
problems, and really opens the door to completely grandios perfectly 
idealistic genericititousness ;)  , a straight-forward OO approach to 
solving the problems would 1) be faster and 2) be easier to read and 
maintain.  I've seen reflection based approaches work, and I've seen 
them bomb.

One might argue that in a container based system that will eventually 
support clustering and scalability doesn't have to worry about savoring 
individual CPU cycles that might be expended needlessly with a 
reflection-based design.  Just add more hardware and the user experience 
will be perpetually preserved.  I would argue, however, that the 
development community will be able to make better use of the product if 
the rate at which additional hardware purchases becomes necessary 
decreases.  The philosophy, IMHO, should be to solve the problems at 
hand with straight-forward OO solutions, and not to focus so much on 
being generic and supporting n * infinity solutions up-front.  Using 
things like the Policy/Strategy pattern at well-selected locations will 
afford the opportunity for pluggability without compromising 
maintainability and performance.

Here are some dramatic results:

*Direct:*

public void init()
{
   customer = new Customer();
}

public void run()
{
   customer.work();
}


*Interface (Polymorphism):*

public void init()
{
  customer = new Customer();
}
public void run()
{
  Person person = (Person) customer;
  person.work();
}


*Reflection:*

public void init()
{
  customer = new Customer();
  try 
  {
      method = customer.getClass().getMethod("work", new Class[0]);
  } 
  catch (Exception e) 
  {
  }
  
   params = new Object[0];
}

public void run() 
{
  try 
  {
    method.invoke(customer, params);
  } 
  catch (Exception e) 
  {
  }
}


With 1000000 of the above code, here were the results:

JDK DirectTest InterfaceTest ReflectionTest
*Sun 1.4* 52 ms 54 ms 543 ms
*Sun 1.4 -server* 26 ms 56 ms 279 ms
*Sun 1.3* 124 ms 128 ms 2168 ms
*Sun 1.3 -server* 41 ms 58 ms 2012 ms
*IBM 1.3* 75 ms 78 ms 2134 ms


Reflection will significantly effect the performance of the system and 
should be avoided for any runtime operations.  Where it should and could 
definitely be applied is with the dynamic generation and compilation of 
code, or generation of metadata.  Otherwise, even at the recommendation 
of Sun, it should be avoided for runtime operations.

Best,

John C. Dale

Leo Simons wrote:

    
Jason Dillon wrote:

      
PS. Can someone write up something about the current state of the 
major component containers out there with a feature blurb... no soap 
boxes, just the facts jack.
        
What, no soap boxes? How on earth can anyone comply with that? ;) No 
wait,
you weren't asking me anyway, were you. Ah, e-mail already typed. Bummer.

= My Opinion =

You should *not* be evaluating component containers. You should save 
this discussion
for a later date and just code your way to 1.0. The basic design idea 
overview
below should show that most of the architectural concepts behind all 
these containers are
very similar. I've been experimenting (no, I will not plug it, you 
will just get confused) with
some reflection that will allow any component written for any of the 
below to run in any
of the other containers, and that is feasible, straightforward and 
performant.

So write your components to plug in whatever you have, do a nice IoC, 
SoC, SAI, AOP
setup, and you will be able to defer refactoring around an external 
container until much later.

But that's my opinion, and I have now said it three times, and your a 
responsible adult
(yep, its a guess, you could also be 11 years old :D). Switching 
soapbox mode off.


= Disclaimer =

Comparing component containers is comparing apples with pears. Avalon 
is by far the
biggest 'generic' project at the moment, for example, but recent 
developments utilize AOP
and interceptor architecture to support a much 'lighter' 
container-component api and
contract. Indeed, picocontainer was started by an avalon elder from 
the firm belief that
things should be simpler and smaller. So to actually evaluate all this 
stuff, you really should
spend a day or so delving into the websites and the code of all these 
projects, and backing
tech like nanning (nanning.codehaus.org) and aspectj 
(www.aspectj.org). I would start
by looking at pico/nano and xwork, then take a look at the tutorials 
for avalon-merlin.
The other projects have a smaller community atm, and I tend to value 
community size
and vibe.

Furthermore, I have strong opinions about stuff, and allegations to 
various projects, hence
this is not an objective overview, even though I tried to make it 
somewhat objective.


= Features/ design idea shorthands =

IoC = Inversion of Control, the idea that an application is controlled 
from the top down
SoC = Seperation of Coccerns, the idea that a class (aspect) should do 
one job and do it well
SAI = Seperation of API from Implementation, the idea that you define 
and code to work
   interfaces
AOP = Aspect Oriented Programming, mostly lightweight nowadays where 
you add a chain
   of interceptors around a method call that can handle orthogonal 
concerns
DecP = Declarative Programming, where you use a declarative-style 
language (usually xml) to
   determine things like component wiring (ie your average tomcat 
config file, generalized)
EBP = Event Based Programming, basically making the inter-object 
method call asynchronous
   and encapsulating such a call into some kind of event object that 
can be queued, modfied,
   etc


= No particular order, incomplete list =

http://wiki.opensymphony.com/space/XWork - IoC, SoC, SAI, AOP, DecP, 
EBP. Nearing
1.0 release. Used in webwork2 (a competitor to struts). EBP very basic 
only. Lean and mean,
but not mature and some client-server web-layer specific assumptions. 
Don't like the XXXAware
interfaces. Very vibrant and active community and many famous peeps 
with J2EE experience
around at opensymphony.

http://www.picocontainer.org/ and http://www.nanocontainer.org/ -
IoC, SoC, SAI, AOP, DecP. 1.0 beta releases. Lean and mean and very 
extensible and
embeddable, developed by smart XP peeps, some stuff already in use in 
some apps, but
otherwise pretty much alpha. I love picocontainer and the way they're 
doing the project.
The dev community is intentionally kept small atm, but many peeps are 
watching this one.

http://plexus.codehaus.org/ - IoC, SoC, SAI, DecP. Container supporting
avalon-framework components. Used to be 
yet-another-novel-avalon-container, but I
think they're growing to be container-component-contract-agnostic. 
Corporate backed
development. Smart guys, not so much focussed on releases as on 
getting all the
functionality they need (which is a lot) in place. Very much a 
pragmatic project.

http://www.jcontainer.org/ - Not yet public container development
(dubbed 'loom' IIRC) by a smart ex-avaloner 'n others. Haven't seen 
any code yet
but my guess is it'll be good. The website says "move along" so you 
prolly should.

http://www.springframework.org/ - haven't looked at in too much depth. 
Seems similar to
nanocontainer and xwork. Think it has one active developer and a beta 
release. Some
smart points made, but too much xml for my taste. Hoping to see some 
of this rolled into
Xwork and/or pico.

http://avalon.apache.org/ - IoC, SoC, SAI, DecP, EBP (EBP for fortress 
only). By far
the oldest 'generic container' project. Big committer base, mature 
codebase, mature
ASF project (which can be a good thing and a bad thing ;) rather 
extensive
'avalon-framework' (comparatively heavy compared to more recent 
developments)
that defines the contracts between a component and a container. Has 3
container projects to consider: avalon-phoenix, a mature microkernel 
design,
avalon-fortress, similar in weight and featureset to something like 
nanocontainer with a
1.0 release (successor to avalon-ecm, the container used in (among 
other projects)
apache-cocoon), and avalon-merlin, a more recent development which we 
_seem_
to be converging on as the successor to all other previously produced 
containers. Merlin
is further described in the email by Stephen Mcconnell. Arguably the 
biggest,
most extensive and most dynamic IoC container implementation around 
(and hence
also the most complex). Re: my earlier blurb on 'geronimo and avalon' 
for more
yadayada.

http://jakarta.apache.org/commons/sandbox/hivemind - haven't looked at 
in much detail, but seems
very similar in scope and setup to avalon at first glance. Framework 
being refactored out
of hibernate, one developer, still in alpha with no releases I think. 
No offense to Howard
intended, but I think he's cut himself a rather big piece of the 
puzzle to recode from scratch
at once. But I am an uninformed whiner, so I'm not going to comment 
further in the hope that
Howard will just eventually see the light and direct his energy 
towards collaboration with the
avalon peeps :P


I am spending too much time on writing messages to this mailing list 
(it is nearly 3am over here).
I promise this is the last message from me for two weeks :D

g'night,

- Leo

--------------090404040005060309040505--