geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geir Magnusson Jr <>
Subject Re: online and offline deployment
Date Mon, 08 Nov 2004 23:29:07 GMT

On Nov 8, 2004, at 1:52 PM, Dain Sundstrom wrote:

> Guys,
> After reading these two massive emails, I feel no closer to  
> understanding what Jeremy wants to change.  Jeremy, I think it would  
> help me if you could summarize what you propose we do differently from  
> what was voted on the week before last and clarify whether your  
> proposal would be in addition to the solution we voted to do or if it  
> would replace that solution.

Maybe be could call you on the phone and explain?


> Thanks,
> -dain
> --
> Dain Sundstrom
> Chief Architect
> Gluecode Software
> 310.536.8355, ext. 26
> On Nov 7, 2004, at 6:36 AM, Aaron Mulder wrote:
>> 	Just to reiterate, I think Jeremy is saying that using the
>> deployer tool for offline install is limited because it doesn't know  
>> what
>> GBeans the server is using for the ConfigStore and  
>> PersistentConfigList
>> and so on.  If we instead actually start the server to do an "offline"
>> deployment/installation, then all the corrct GBeans will be running  
>> and
>> that is no longer an issue.
>> 	An alternative would be for the deployer to inspect the server's
>> configuration when it starts, and load every dependency from the
>> immediate parent of the module to be deployed up through the "root",  
>> and
>> that should identify the correct ConfigStore and PersistentConfigList.
>> But this is tricky too, since how would it know what ConfigStore to  
>> load
>> the configurations out of (including the configuration for the
>> ConfigStore, aargh!).  In the end, I suspect this depends on how
>> server.jar was packaged, and if you plan to start your server with
>> start-my-server.jar instead of server.jar then I don't know how the
>> deployer would know that, so I don't know where it would get the  
>> original
>> ConfigStore reference from -- perhaps we'd need to give it an option  
>> to
>> identify your server startup JAR.  But I think this would still fail  
>> if
>> the server was running (since you'd probably clash for ports trying to
>> load some of the services between ConfigStore and application), so  
>> it's an
>> "offline" deploy in name only.
>> 	Another option is that we can provide a tool that works 100% for
>> the default server configuration  
>> (LocalConfigStore+FileConfigurationList).
>> But it would not work in the face of customizations to the 2 core
>> components: if you swap out your LocalConfigStore, then the tool  
>> would not
>> work (it would install into the wrong place), and if you swap out your
>> PersistentConfigurationList then the tool would be unable to mark any
>> module to be started.  If we wanted to, we could make offline deploy  
>> tools
>> available for different combinations of those GBeans, or give you a
>> procedure to build a new deploy tool from an old one.
>> 	The main reason I feel that this is important is that most other
>> products support it.  Generally if you copy a new EAR over an old one
>> while the server is not running, and then start the server, the new
>> version of the EAR will deploy on startup.  (Tomcat 5 in the one I can
>> think of that doesn't do this).  I just hate to tell people that  
>> things
>> that used to work won't work any more if they move to Geronimo.  On  
>> the
>> other hand, I think this behavior was mostly implemented via a hot  
>> deploy
>> directory, so if we provide a GBean for a hot deploy directory, then  
>> maybe
>> we don't need a offline deploy tool at all (beyond for building the
>> server).
>> 	And I guess the last issue is related.  In the long run, it will
>> be nice/necessary to have some kind of packaged-configuration-handling
>> features, in the deploy tool or another tool:
>>  - extract a CAR file from an entry in a server's ConfigStore
>>  - sign a CAR file (either in the server's ConfigStore or as a file)
>>  - transfer a packaged configuration directly from one server to  
>> another
>>  - deploy a CAR file into a server
>> Aaron
>> On Sat, 6 Nov 2004, Jeremy Boynes wrote:
>>> As promised Thursday, here are the details of my concerns about  
>>> mixing
>>> offline and online deployment.
>>> My concerns on this issue stem from how we package GBeans together  
>>> for
>>> use by the kernel. Rather than handling them one-by-one, Geronimo  
>>> uses
>>> the notion of a pre-canned Configuration which contains a number of
>>> GBean instances and the classpath information needed to load them.
>>> Configurations can be loaded by the kernel and when started bring all
>>> the GBeans they contain online together.
>>> A key feature of Configurations is they are portable between  
>>> different
>>> Geronimo installations - specifically a Configuration can run in any
>>> Geronimo kernel that can resolve its dependencies. This is less  
>>> critical
>>> for the single-server mode we have now but is very important as  
>>> Geronimo
>>> scales to clustered or grid configurations - it allows us to  
>>> efficiently
>>> move applications between the servers on demand.
>>> This also has benefits where change management is important, such as
>>> business critical installations. For example, a Configuration can be
>>> built and signed in a test or integration environment and moved
>>> *provably unchanged* though the test, stage and release to production
>>> process. Alternatively, an OEM can release an application to channel  
>>> as
>>> a signed Configuration, end-users can have the assurance it has not  
>>> been
>>> tampered with, and the OEM can reduce costs by reducing problems  
>>> caused
>>> by variations in the end-user environment.
>>> In the kernel, the process of loading and unloading Configuration is
>>> handled by a ConfigurationManager that uses ConfigurationStores to  
>>> store
>>> them. The store exposes a simple API for installing and uninstalling
>>> Configurations and for retrieving them so they can be loaded. We  
>>> have a
>>> simple LocalConfigStore implementation that uses the local  
>>> filesystem to
>>> store them; other implementations are possible using different
>>> persistence approaches such as databases, LDAP or proprietary
>>> configuration management systems.
>>> The deployment system in Geronimo is the interface between  
>>> user-domain
>>> artifacts such as J2EE modules (EARs, WARs, etc.) or deployment plans
>>> and the configuration management system described above. It  
>>> essentially
>>> combines modules with plans and generates Configurations.
>>> It comprises three parts:
>>> * External interfaces such as the command line tool, console or  
>>> JSR-88
>>>    provider that get the modules and plans from the user
>>> * ConfigurationBuilders such as EARConfigBuilder and
>>>    ServiceConfigBuilder that do the combination and produce the  
>>> target
>>>    Configuration
>>> * Back-end interfaces that store the Configuration either in a
>>>    ConfigurationStore or as an output file
>>> The ConfigurationBuilders are GBeans and run inside a Geronimo  
>>> kernel.
>>> Apart from ease of implementation, they also have access to the
>>> resources provided by that system - for example, they can use the
>>> Repository to load classes during processing, and they can use the
>>> ConfigurationManager to load other Configurations that the target  
>>> may be
>>> dependent on.
>>> To support online deployment, we run a deployment system inside the  
>>> same
>>> kernel as the J2EE server - it is actually part of the
>>> org/apache/geronimo/Server Configuration although work is progress to
>>> allow it to be run as a separate dependent configuation.
>>> The JSR-88 provider interacts with this deployment system to fulfill  
>>> the
>>> spec requirements for distribute, start, stop, undeploy etc. For
>>> example, during a distribute operation the module and plan are  
>>> passed to
>>> the deployment system, it uses an EARConfigBuilder to produce the  
>>> output
>>> Configuration, which it then installs in the target  
>>> ConfigurationStore.
>>> A JSR-88 start operation causes the Configuration to be loaded from  
>>> the
>>> store and then started.
>>> However, this leaves us with a chicken-and-egg problem. The online
>>> deployment system above is itself part of a configuration - how do we
>>> build that configuation?
>>> To solve this, and because it seemed generally useful, we built a
>>> standalone offline deployment system. Run from the command line, this
>>> would take module + plan and produce a Configuration. To reuse as  
>>> much
>>> of the configuration building infrastructure as possible, it boots an
>>> embedded Geronimo kernel and loads a Configuration containing just  
>>> the
>>> deployment system. As a running kernel, it also provides access to a
>>> Repository and ConfigurationStore that the ConfigurationBuilders can  
>>> use
>>> to resolve dependencies (including dependencies on other
>>> Configurations). However, these are *its* Repository and
>>> ConfigurationStores and *not* those from the target server.
>>> To cheat our way around the chicken-and-egg problem we took the  
>>> simple
>>> but expedient solution of having the standalone deployer and the  
>>> default
>>> server use the same type and location of store and repository. Then,  
>>> by
>>> simply telling the standalone deployer to install a configuration  
>>> into
>>> its own store it would also be available to the default server
>>> configuration. This is a hack, pure, simple and effective.
>>> When we introduce any additional complexity into the situation, then
>>> this hack starts to break down. For example, if the user adds a
>>> database-based ConfigurationStore to the server (for example, to make
>>> GBean state persistence more reliable) then the standalone deployer
>>> would not be able to install the generated Configuration into that  
>>> store.
>>> All things considered, I think having options in the standalone  
>>> deployer
>>> that rely on it sharing the same type and location of Repository and
>>> ConfigurationStore will lead to obscure behaviour and strange  
>>> behaviour
>>> as soon as we progress beyond the most basic default configuration.  
>>> That
>>> is why I voted at the start for "2 simple tools rather than one  
>>> complex
>>> one."  Going further as has been proposed and coupling the standalone
>>> deployer to the internal implementation of the
>>> PersistentConfigurationList seems like pouring gasoline on the fire.
>>> I have been portrayed on this list as being alone in my opinion but I
>>> will point out that in the initial vote Eric LeGoff, Aaron Mulder and
>>> David Jencks also voted for 2 tools (as opposed to Peter Lynch,  
>>> Davanum
>>> Srinivas, Hiram Chirino and Bruce Snyder who voted for one); Dain
>>> Sundstrom voted for one tool, but wanted another to support the
>>> functionality we have to output Configurations as jars ("that is  
>>> another
>>> tool for another day" but we need it now to build the server) which
>>> sounds like two tools to me.
>>> After that vote, Aaron proposed and attained consensus for a single
>>> tool. The syntax is simple enough and mirrors the JSR-88 API making  
>>> it
>>> ideal for online deployment (which is all JSR-88 supports).
>>> However, during implementation Aaron ran into the issues described  
>>> above
>>> and on the thread from 11/4/04 when trying to support the offline  
>>> mode
>>> not covered by JSR-88. These are clearly technical issues which we  
>>> need
>>> to resolve. To facilitate that, Aaron proposed to commit his work so
>>> that all could see and discuss; he and I were promptly and  
>>> unjustifiably
>>>   flamed by some members of the community.
>>> Since Thursday he has committed this code and I think we need to  
>>> review
>>> where we are. My belief is that the online side is fully implemented,
>>> that the standalone deployer works as before (package option), and  
>>> the
>>> big remaining issue is the one described above where someone is  
>>> trying
>>> to "deploy" applications to an offline server.
>>> In this message
>>> I wrote that you could distribute to your heart's content; this was
>>> wrong. The discussion with Aaron highlighted that the problem about
>>> store type and location applies to distribute as well as the other
>>> operations. It looks like the only thing you can reliably do offline  
>>> is
>>> package a Configuration for later use.
>>> I would suggest then, rather than the --add option I proposed for the
>>> server we instead have a --install option which boots the server,
>>> restarts all previously running configs and installs the new one. The
>>> offline usage would then be:
>>> java -jar deployer.jar package foo.war foo-plan.xml
>>> java -jar server.jar --install
>>> This also provides a simple mechanism for deploying once and running
>>> everywhere: the output configuration can be installed in multiple  
>>> places
>>> as easily as one.
>>> The issue with this is that it fits the admin's view better than the
>>> developer's. However, I continue to believe the:
>>>    start server
>>>    repeat
>>>       write code
>>>       build (with distribute/start to online server)
>>>       test
>>>    until app works or it's time to go home
>>> cycle is what most developers use and that Aaron's changes (in
>>> conjunction with the existing Maven plugin) have made it easy for  
>>> them
>>> to work that way. They are not really interested in fancy offline
>>> deployment tricks.
>>> To support carrier-grade configuration management, clustered and grid
>>> environments and OEMs, I believe we need an effective way of  
>>> generating
>>> pre-packaged configurations using requires Maven/Ant plugins
>>> that can be used in the release process, tools like Aaron's that an
>>> administrator can use from the command line, and mechanisms for
>>> installing them in and for transporting them between between servers.
>>> I think we are very close to achieving this and if we can address  
>>> these
>>> last issues then Geronimo will be acceptable to both the developer
>>> community and to serious IT decision makers.
>>> --
>>> Jeremy
Geir Magnusson Jr                                  +1-203-665-6437

View raw message