geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Jencks <>
Subject Re: Deployment architecture
Date Mon, 03 Jan 2005 21:30:56 GMT

On Jan 3, 2005, at 11:53 AM, Dain Sundstrom wrote:

> On Jan 3, 2005, at 11:04 AM, David Jencks wrote:
>> Combining a couple of responses...
>> On Jan 3, 2005, at 9:28 AM, David Blevins wrote:
>>> Just a couple curiosities.
>>> Would moving the coupling between module builders to be coupling on 
>>> work objects achieve much as far as actually decoupling the 
>>> deployment system, or would it just introduce another brittle layer 
>>> in the existing coupling?
>> I don't know:-)  I think the main decoupling effect would be that 
>> each builder would be able to say "someone needs to process this" 
>> rather than "you, in particular, need to process this".  It looks to 
>> me as if the current system is getting more and more classes that 
>> need to know specifically about one another and call specific (often 
>> static) methods on each other.  I'm wondering if there is some way to 
>> make this more uniform and generic and perhaps extensible and in 
>> particular to easily support web service and portlet deployment.
> IMNSHO this is caused by our usage of XML beans.  When I talked to the 
> XML beans guys at ApacheCon they told me to put an abstraction layer 
> between XML beans and our deployment code.  This allows them to 
> innovate and us to have well defined objects with interfaces.  If we 
> had interfaces (and normal objects) for our metadata, we would be able 
> to create generic metadata processors.  I personally believe that the 
> majority of the "ugliness" of our deployment code is the lack of 
> abstraction (flexibility) of our metadata objects.

Can you give me some more specific idea of what you are talking about 
and how it might work?  I'm intrigued but really have no idea what you 

>>> Any ideas on how we would dictate the order of the deployment chain?
>> Either writing a little class like the ejb interceptor builders or by 
>> some kind of deployment descriptor.  If each element is a gbean, 
>> linking them into a chain is pretty easy with our current xml based 
>> gbean deployment system.
> Why do we need order?
>> On Jan 3, 2005, at 9:49 AM, Dain Sundstrom wrote:
>>> I'm a bit confused about how you envision this working, so please 
>>> take my comments with a grain of salt :)
>> Me too, at this point I'm speculating:-)
>>> On Jan 3, 2005, at 1:05 AM, David Jencks wrote:
>>>> The current architecture of deployment might be considered to have 
>>>> some limitations.  It is not clear how to extend the system to 
>>>> deploy more artifacts such as web services and portlets, let alone 
>>>> artifacts we don't know about yet.  There is also a growing web of 
>>>> dependencies between module builders to take care of ejb 
>>>> references, connection factory references, and bits such as 
>>>> security and naming.  I also think there may be situations where an 
>>>> incompletely processed artifact does not cause a deployment error.  
>>>> I wonder if there is a simpler more extensible architecture.
>>>> What I'm thinking of has two main new features:
>>>> 1. a chain of  builders with only one method, "build"
>>> Do you really mean a chain?  The word chain implies an order to me, 
>>> so is there an order you are thing of?  If not, how about "set" or 
>>> "bag".
>> Definitely an ordered chain of "interceptors".  All deployments would 
>> happen by feeding stuff into the single topmost interceptor.
> Why do we need order?  I thin it would help me if you could give an 
> example of what the order would look like.
>>> 2. some sort of "deferred work" object that allows one builder to 
>>> ask another one, farther down the chain, to do some work for it.
>>>> Lets look at how builders interact with each other:
>>>> One type of interaction is for a builder to add extract some 
>>>> information from the current deployment plan and cache it in the 
>>>> deployment context for use by itself or other builders later.  For 
>>>> instance, the connector builder figures out the activation spec 
>>>> metadata and puts it in the context for use when deploying message 
>>>> driven beans.
>>>> This type of interaction can be  handled by a chain of builders by 
>>>> simply having two builders: the first one adds the shared info to 
>>>> the context, and the second one(s) use it.
>>> If I have this correct, you are suggest we switch from a push 
>>> (cache) system, to a pull system.
>> I don't think so, or at least not entirely.  I am suggesting that the 
>> "init context" method would be one interceptor that adds stuff to the 
>> deployment context so it can be used by any later interceptor.  
>> However, instead of say the ejb builder directly calling the naming 
>> builder with snippets of info to build the read-only context, it 
>> would just add an object to the context to be processed later by the 
>> naming builder.  The advantage I see here is that the ejb builder 
>> doesn't need to know anything about who is going to do the work for 
>> it, nor does it need a reference to the naming builder.
> If we had normal metadata object, the objects themselves could perform 
> the processing located in our "static" methods.  Anyway, isn't it the 
> job of the EJB builder to provide ejb-refs?  Of course he can delegate 
> it to another object, but doesn't the buck stop with the ejb builder.  
> Said another way, shouldn't that be an internal detail of the ejb 
> builder?
>> I think I remember someone mentioning the idea of having some kind of 
>> builder registry so that a builder when faced with some plan bits it 
>> doesn't deal with directly could ask the registry to find the 
>> appropriate builder and process it.  I think this deferred work idea 
>> provides the same functionality without a central registry.
> I'm not sure I like that also.  My thoughts have revolved around 
> having a single builder responsible for a module type and child 
> builders that can add stuff the deployment but are not the main 
> builder.  For example, wars are currently handled by the JettyBuilder 
> which means that it is this builder that is responsible for coping the 
> files from the war archive into the deployment context, setting up the 
> bulk of the class path, and the module GBeans.  Then if we had a 
> portlet builder, it would get a chance to add to the classpath and add 
> additional gbeans to the the deployment.  I envision this being 
> implemented by the web module builder having a collection reference to 
> child web module builders, and iteration over the set in each build 
> method.  Is there something I'm missing that makes this complex?

Well, right now the ear builder has references to a bunch of other 
builders that it has to explicitly delegate various work to.  Your 
proposal for the web builder is similar, in that it will have a set of 
sub-builders that at various times it explicitly calls.  With an 
interceptor chain, you wouldn't need these references at all: the ear 
builder would simply decompose ears into a bunch of modules, and the 
other builders would find and work on them.  The web builder would just 
work on the web.xml, and the portlet builder would just add stuff to 
the context.  One way of looking at this is that with the interceptors 
we are building a high level domain specific language for deployment, 
and setting up the interceptor chain is programming deployment in that 
language.  Instead of java code maintaining and using references from 
one builder to another, we set up the interceptor order to do the same 
work for us.   Of course the current reference-based deployment system 
can also be regarded as a domain specific language.  I think it might 
be harder to understand since it has many more references and can't be 
written down linearly.  I'm by no means convinced that my idea is good, 
but every time I've switched to an interceptor based architecture it 
has really improved the power and simplicity of the system.
>>> I'm not sure that will work in the case of an EJB ref.  An EJB ref 
>>> has matching rules, that really need holistic view of all EJBs in 
>>> the deployment unit.  Off the top of my head we have the following 
>>> precedence rules:
>>> * Exact ejb-link specification of EJB module and EJB name
>>> * No module in ejb-link, but an EJB in the current module has the 
>>> same name as the ejb-link name
>>> * No module in ejb-link, but only one EJB in the EAR that has the 
>>> same name as the ejb-link name
>>> * No ejb-link, but only one EJB has the same name as the type and 
>>> interfaces of the ejb-ref
>>> For most ejb-refs, you need to need to inspect every EJB in the ear.
>> I should be a bit more specific:-)
>> Lets say we're deploying an ear with 2 ejb modules.  Here's a sketch 
>> of what happens:
>> ear-builder: detects modules in the ear and adds 2 ejb-work objects 
>> to context
>> ejb-init-builder: for each ejb-work object, adds name of ejb to 
>> context (i.e., does the init context work from current ejb-builder)
>> (other stuff, like connector deployment...)
>> ejb-builder: for  each ejb in each ejb-work object, starts 
>> constructing the ejb container gbeans.  It adds a naming-work object 
>> to the context that will fill in the read-only-context attribute of 
>> the gbean when it is processed.
>> (other stuff)
>> naming-builder: for each naming-work object in the context, it uses 
>> builds the read-only context and sets the GBeanData attribute value.
>> (other stuff)
>> serializer: checks that there are no unprocessed work objects, and 
>> serializes the configuration
> My gut says, this is way complex and will be hard to debug.  The is 
> mainly since there is no way to know who is responsible for a piece of 
> data.  At the end, you have an extra piece of data, and you don't know 
> if it was optional, if it is bad data, or it was good data and the 
> expected builder was not online.  Just my gut.

You could well be right, but I'm not sure it is any different from what 
we have now.
>>>> Another type of interaction is for a builder to extract some 
>>>> elements from the spec and/or vendor plan and immediately ask 
>>>> another builder to process it.  For instance, ejb and resource 
>>>> references are immediately processed by the naming builder, the 
>>>> vendor security descriptor elements are immediately processed by 
>>>> the security builder, and dependency and gbean elements are 
>>>> processed by the service builder.
>>> Why do we need "builders" for security, naming, and gbeans.  Can't 
>>> we get buy with a utility class instead of a full blown service?
>> That's what we have now.  It works, but I'm not thrilled about the 
>> extensive use of static methods.  We also have the client builder 
>> needing a reference to the connector builder.  We want to be able to 
>> deploy connection factories and admin objects (datasources and 
>> topics/queues) from any plan, so at least the ejb, web, and ear 
>> builders would need references to the connector builder as well.  I'm 
>> worried we will get an impenetrable web of connections between 
>> builders that is difficult to set up or maintain, and wondering if it 
>> can be simplified.
> I don't like them either.  IMNSHO these static methods belong in the 
> metadata objects, but since we don't have metadata object they end up 
> in helper classes.  I think once you introduce real metadata object, a 
> huge amount of complexity will fall out of the deployment code.

Again, I'd really like to know more about what you are thinking of and 
how it would work.

>>>> Most of these are used to construct a gbean attribute value which 
>>>> is not used further during deployment.  So, perhaps the builder 
>>>> needing the work done could construct a "deferred work" object 
>>>> containing the element to be processed and the gbean and attribute 
>>>> name, and add this to the context.  Then the, e.g., naming builder 
>>>> could look in the context for deferred work objects that it 
>>>> understands, process them, and set the appropriate gbean attribute 
>>>> values. If any deferred work objects were left over unprocessed 
>>>> when it came time to serialize the gbean state, we would know there 
>>>> was a deployment problem.
>>> Ya, you lost me here.  Can you be more specific on where I would use 
>>> "deferred work" and how it would be implemented?
>> Is the ejb example above detailed enough?
> Yep.
>>>> The dependency elements in the plans are used to construct the 
>>>> classpath, which is needed during deployment. Each builder could 
>>>> extract these elements and put them in deferred work objects: in 
>>>> this case the processing would not set a gbean attribute value, but 
>>>> would result in adding to the classpath.
>>> Not sure how that is better then what we have today.
>>>> The ear deployer might fit well into this scheme.  Instead of 
>>>> calling specific module builders for each type of module it 
>>>> recognizes, it could bundle up the info for each module into a work 
>>>> object that it adds to the context: the current module builders 
>>>> would then look in the context for the work objects they 
>>>> understand.
>>> Again, I'm not sure how that is better, but I think that would be an 
>>> easy change to make.  Instead of having an 4 builders we have a set 
>>> and ask each one if it can handle a specific module (or give it all 
>>> modules and say do what you can).  I'm just now sure it buys us 
>>> much.  I most afraid that we will loose the ability to have good 
>>> error messages since we don't know who is supposed to be responsible 
>>> for a module.
>> I'm proposing that we shift from the model of deployment components 
>> having a list of builders that they iterate over in hopes of finding 
>> one that works, to a model  where you just throw stuff on a single 
>> chain and look to see if everything got done at the end of the chain. 
>>  I don't see how this affects our ability to improve our currently 
>> uninformative error messages: either way, if there's a builder that 
>> recognizes that it should be able to process something, it can 
>> provide a specific error if it fails, and if there isn't, we don't 
>> recognize something so its hard to say what we should have done.
> I'm getting the feeling that this is just different, but we have the 
> same problems.

Probably, but with a single well defined end of the chain it might be 
easier to check for unprocessed bits.  I don't see this as being a 
significant argument either way.

david jencks
>>>> Right now I'd characterize these ideas as speculation and wonder if 
>>>> anyone else thinks they are worth pursuing further.  I also wonder 
>>>> if limiting the objects in the deployer chain to a single method 
>>>> has too little structure and giving them several methods to be 
>>>> called in a specified order, as with the current architecture, 
>>>> would provide better organization without limiting useful 
>>>> functionality.
>>> I did consider using a single "doIt" method when working with 
>>> deployment last time, but the problem is you trade coupling in 
>>> methods (interface) for complexity in the parameters.  For example, 
>>> all of Java could be rewritten with each class having a single 
>>> method doIt(Map data).  I know that you are not implying that we go 
>>> that far, but this demonstrates the complexity trade-off.  I suggest 
>>> something in the middle, a few methods with a few mildly complex 
>>> parameters :)
>> I haven't gotten that far yet:-)  To me the question is whether it's 
>> clearer to put all the, e.g., ejb processing methods in one class and 
>> possibly have lots of builders that do nothing in many of the methods 
>> or have more than one "ejb builder" class, one for each method.  I 
>> suspect that having 2 or 3 methods will be more convenient but I 
>> don't have a firm opinion yet.
> I think we should put the responsibility into one class and allow the 
> implementation to choose how to distribute the complexity of that 
> responsibility.
> -dain

View raw message