Mailing-List: contact directory-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List"
 <directory-dev@incubator.apache.org>
Message-ID: <40E6049F.4080601@apache.org>
Date: Sat, 03 Jul 2004 02:58:07 +0200
From: Stephen McConnell <mcconnell@apache.org>
User-Agent: Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.6b) Gecko/20031208
MIME-Version: 1.0
To: Apache Directory Developers List <directory-dev@incubator.apache.org>
Subject: Re: [eve] merlin wrappers
References: <1088780218.16055.7.camel@fermi.trunk.joshua-tree.org>
	 <40E5AD68.4020804@apache.org>
 <1088803962.17994.4.camel@fermi.trunk.joshua-tree.org>
In-Reply-To: <1088803962.17994.4.camel@fermi.trunk.joshua-tree.org>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Alex Karasulu wrote:

> Vince, Steve,
> 
> <snip/>
> 
>>>We can still benefit from this to allow hot
>>>swaping if the Avalon wrapper components are in separate jars.  Is this
>>>a correct assumption?
>>
>>The avalon components "could" be in separate jars but they will be 
>>sharing the jars containing the apis and pojos.  This means that the 
>>only thing you gain is the ability to swap out the adapter (and I really 
>>can;t see why that would be useful).
> 
> Not exactly.  Your adapter component, which from now on I'll just for
> simplicity refer to as the adapter, may or may not use the default POJO
> implementation.  The "adapter" may not be an adapter at all or may be an
> adapter to a component implementation that is not the default POJO
> implementation in the frontend.jar.  So the frontend.jar falls back to
> being the api jar in situations where the default POJO is not being
> used.
> 
> So the potential to swap out adapter components is still there.  One
> version of the adapter may implement interface Foo, and it may wrap the
> POJO Bar which implements Foo as well.  Another version of the adapter
> may wrap another object of class Xyz which implements Foo.  So we can
> swap out one component for the other and be ok hot swapping this way.
> 
> Effectively everything in the frontend.jar by containing POJO
> implementations with their service interfaces is an API.  Because at the
> end of the day the component developer chooses whether or not some
> default implementation is worth using for them or not.  This API can be
> used by anyone including us while writing wrappers for various
> containers.  We may opt to use the default POJO provided if we want to
> as a part of the API.  It is completely up to the framework specific
> component developer.
> 
> 
>>The main interest in hot-swapping is the ability to bring in a different 
>>implementation and once established, re-sync a service to point to the 
>>the new implementation (without taking down dependent services).  Merlin 
> 
> 
> Not a problem at all.
> 
> 
>>3.3.1-DEV provides support for the loading and unload of containers 
>>dynamically, add, the ability to modify and remove component models 
>>without a restart (there still of another layer needed to handle service 
>>availability).
> 
> 
> That's great and should be very exciting. 
> 
>>What this means is that we want to be able to swap out eve version X.Y 
>>with eve X.Z - and that means demounting an implementation classloader 
>>containing the pojo classes while maintaining the api classes in the 
>>classloader (because these are referenced by consumers).
> 
> We can still have this by keeping a single frontend.jar with the service
> interfaces, and their POJO implementations in the same jar.
> 
> 
>>>The second thought was what the heck am I going to do with the different
>>>conf/ directories I have for integration tests on the Merlin wrappers
>>>using Merlin-Unit.  Right now having each component in its own project
>>>makes it easy to have its separate conf/ directory to test the component
>>>with.  The top level compoent for the frontend called the 'frontend
>>>component' will have the final block.xml for the server's
>>>configuration.  What are your opinions and thoughts here?  
>>
>>My current thinking is that we should structure thing as follows:
>>
>>    /frontend
>>       /api <------ * absolutely minimal external dependencies
>>                    * interface, exceptions
>>                    * immutable data objects
>>       /impl <----- * pojos, implementation classes, resources
>>       /merlin <--- * definition of a container that exports
>>                      services exposed by the api and handles
>>                      deployment based on internal component
>>                      definitions
>>
>> From the point of view of a classloader what this means is:
>>
>>     |-------------------------|
>>     | eve api                 |
>>     |-------------------------|
>>     | merlin adapter          |
>>     |-------------------------|
>>     | eve impl                |
>>     |-------------------------|
>>
>>WDYT?
> 
> 
> This does look rather nice.  I especially like the part about the api
> jar having minimal dependencies.  However I don't think this outwieghs
> at least in my mind the points I've made above.

If that doesn't - the following will.

Generally speaking - a good API does not expose dependencies on other 
libraries.  Inversely - and implementation can go to town and use 
whatever it needs.  The issue is that you don't control the classloader 
your running within.  This has some very heavy implications.  Assume for 
a moment that you expose commons lang 2.0 and avalon framework 4.1.5 as 
part of your API.  Let's also assume that (a) your running as an 
embedded component inside another system that you know nothing about - 
and lets assume that the embedder is running framework 4.2, secondly, 
lets assume that your running a plugin of some kind inside eve that 
assumes commons lang 1.0.

OK - so a runtime - you implementation classes will never see framework 
4.1.5 because the 4.2 classes are in a parent classloader - which means 
that potential differences may exist (e.g. bug fixes in 4.2 over 4.1.5 
that may create a side effect you never tested for) - but also at the 
other end of the spectrum - the plugin your running is assuming commons 
lang 1.0 has been subverted by eve because eve is assuming 2.0 and even 
worse - if the plugin also assumes framework 4.1.5 its also being 
subverted by the system containing eve. And throw into this any classes 
in the containers bootstrap classloader (which again is totally outside 
of your control).

But it gets worse - if you expose a bunch of external classes in you API 
- it's like getting married to all of them collectively (including all 
of the api exposed by the apis your exposing and the api exposed by the 
dependent apis).  I.e. it's like getting married concurrently, paying 
alimony up-front, and no honeymoon.

This is all simply due to the fact the the JVM goes hunting for classes 
up the classloader chain before looking locally (for reasons of 
security).  That's means that you loose the contract of assured 
computation integrity the moment you expose something.  And this 
situation will not change until the java language introduced version at 
the level of classes and interfaces (and that's not going to happen 
anytime soon).

But there is a solutions ..

   |------------------------------------|
   | API - minimize risk by             |
   | minimizing dependencies - i.e.     |
   | don't let the embedding context    |
   | screw up your assumptions          |
   |------------------------------------|

The API should deal with the challenge of providing adequate 
functionality while at the same time minimizing the risk resulting from 
exposure.  One way of dealing with the dilemma is to break an API into a 
API/SPI pair (which I'm not advocating for eve at this time).  Typically 
the API is there for the mainstream users but the SPI provides a richer 
set of interfaces dealing with perhaps privileged functionality such as 
management and/or sub-system replacement.  For example - the Merlin 
system can be view in terms of its API, SPI and Implementation:

http://www.apache.org/~mcconnell/site/products/runtime/javadoc/index.html

If you look at the three links referenced by the above page - you will 
see a distinct difference between the three levels ... but most 
importantly, the only non-Avalon dependency in the entire API/SPI suite 
is the JRE.  This means that the risk of conflicts are significantly 
reduced. The same applies to the James Mailet API - zero dependencies on 
anything other than James defined interfaces and the Java runtime.

So - expanding my earlier picture a little bit ...

      |-------------------------|
      | eve api                 |
      |-------------------------|
             ^
             | publishes services into
             | a foreign and hostile environment
             |
      |-------------------------|
      | merlin adapter          |
      |-------------------------|
             |
             | establishes, isolates,
             | protects, modifies, evolves,
             | replaces, terminates,
             | etc.
             V
      |-------------------------|
      | eve impl                |
      |-------------------------|


> Although this does not look as well structured I'm leaning more towards
> the following:
> 
> /frontend
> /merlin
> 
> This makes me feel like I'm bouncing between extremes.

Doing a API/IMP separation with an adapter in-between is IMO a 
conservative strategy that achieves the goal of simple structure while 
maintaining integrity.

My 0.02.

:-)

Cheers, Steve.

-- 

|---------------------------------------|
| Magic by Merlin                       |
| Production by Avalon                  |
|                                       |
| http://avalon.apache.org              |
|---------------------------------------|