geronimo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: [RT] blue-sky container
Date Tue, 09 May 2006 10:55:17 GMT
Hi Neeme

On 5/9/06, Neeme Praks <neeme@apache.org> wrote:
>
> Hi all!
>
> My day-job company is in the process of refactoring our in-house-built
> "middleware" into "something better"(tm). That "middleware" is currently
> using plain Tomcat as a container and Spring for "component
> configuration/wiring", publishing its services over SOAP. Due to
> increased requirements (uptime, clustering, monitoring,
> ease-of-maintenance, etc), we are looking into a major architecture
> redesign.
>
> Functionally, the "middleware" is a /telco-grade/ piece of software that
> is used to build services that interface with the (mobile) operator
> systems (billing, SMSC, positioning information, etc) and provide a user
> interface over a variety of channels (wap, web, SMS, MMS, etc).
>
> As an apache contributor, I would love to use (read: push ;-)) Geronimo
> as the base-container for this work. However, I'm not too familiar with
> the internals of Geronimo - so I would like to ask you to give me some
> feedback about the feasibility of this plan.
>
> I've written up my own blue sky vision/ideas for that future container
> (see attachment). It is lacking in details, but I guess you get some
> sort of fuzzy feeling about the thing I'm trying to get at :-)
>
> The aim could be summarized as: build a container that would be
>  * simple for developer (no EJBs, stick to POJO's as much as possible)
>

FWIW the XBean kernel is a pure POJO kernel capable of wiring and deploying
anything at all; we're deploying right now ActiveMQ, Jetty, OpenEJB,
ServiceMix and XFire with it today - all using different kinds of deployment
models and features - but the underlying core is the same - POJOs all the
way up


 * simple for deployer/upgrader (container support for configuration
> management)
>

We still need to do some work on XBean in this area but already we have a
really super simple deployer/undeployer available. e.g. ServiceMix
implements the full JBI based deployment model using JMX & Ant tasks using
XBean - though we can improve this

 * simple for Operations&Maintenance people (container support for
> monitoring, alarms)
>

Yah. It also needs to be distributed too. One option is this...

http://lingo.codehaus.org/JMX+over+JMS

i.e. JMX over JMS so we can publish/subscribe to events etc


 * easy to cluster (HA and load-balancing, no state in middle tier) in
> a reliable way (we have been throwing around the idea of routing all
> method calls over JMS)
>

If you want to take any POJO and cluster it over JMS to make a simple &
powerful computing grid, just a little bit of Spring.xml and this library
does the trick nicely...

http://lingo.codehaus.org/Clustering

creating a distributed WorkManager or implementation of some POJO service is
really trivial (it looks and feels just like Spring Remoting)


I know that everything is possible in the world of software development
> (given enough time and money :-)) so I would narrow my questions a bit:
>  * is this thing a good fit, to be built on top of Geronimo? I guess yes.
>

Yes! :)


 * maybe you can also give some arguments as to why Geronimo is a good
> fit for this?
>

There's a community of like minded developers building & using this stuff to
do similar kinds of things in different areas so there's lots of reuse going
on

 * and, going one step further, could you give some pointers as to
> how/where to start.
>

A good start is maybe surfing XBean and Lingo

And, a final question: would you be interested in at least some of these
> features? As far as I can see, most of the stuff outlined in my blue-sky
> vision is not really specific to our company/industry, it is applicable
> to java development/deployment in general. So, there could be interest
> also from the broader community in implementing this beast and our
> company might be interested in contributing back the generic parts of
> our work.
>
> What do you think? ;-)
>

Sounds good to me :).  A few more observations below (with much snipping of
interesting stuff along the way...)

>  an artifact
>
> Piece of functionality packaged in a way that is ready for deployment. Consists
> of:
>
>    - version, unique identifier and type (maven2 metadata)
>
>
FWIW XBean has some support for putting the classpath or maven metadata in
the XML configuration file itself, then allowing the kernel to boot up a
classpath (downloading the necessary maven jars first) then applying the
configuration in the right class loader. The classloader integration is
there; the maven 2 integration needs a little bit of work but its really
close


>    - compiled java classes - in the future this can be optional, as we
>    have sources anyway. But initially it is more convenient to just use
>    pre-compiled classes.
>    - java sources - used primarily for documentation and bug hunting,
>    but can also be used for compiling
>    -
>
>    JavaDoc <http://wiki/JavaDoc> documentation - can be generated from
>    sources
>    - any metadata about the code that cannot be derived from the java
>    classes and sources
>    - dependency declarations
>
>
Agreed. FWIW I'm hoping we can all adopt this kinda annotation based
metadata for dependency injection and lifecycle support to avoid us having
compile time dependency on  lifecycle APIs like GBean or Spring...

http://xbean.goopen.org/annotation-based-dependency-injection.html

Proper artifact versioning is very important - it should be possible to
> determine the level of compatibility between any two versions by just
> comparing the version numbers.
>
> The whole concept is built on top of Maven2 project descriptors.
>
Agreed!

CORE
>
> The function of CORE
>
> Publish a set of components (interfaces) to:
>
>    - the external world - (remote) clients can use services by
>    including:
>       - a client library - publishes the component types
>       (interfaces) that are publicly available
>       - a transport library - implements the details of accessing
>       the (possibly remote) implementations over
>       JMS/RMI/choose-your-own-transport. The simplest transport would be "no
>       transport" - all services would be available inside the same JVM.
>
>
Agreed. FWIW I'd like to enhance Lingo to deal with ServiceMix remoting
nicely; then we'd get a gazillion transports for free like
FTP/files/email/jabber/IRC/SMS etc - together with various features like
implementing the Enterprise Integration Patterns and so forth

http://incubator.apache.org/servicemix/servicemix-eip
>
> CORE can also be viewed as a "cloud" of middleware nodes in a cluster.
>
>    - Web applications and WSG would be examples of external clients
>    that run outside the CORE.
>    - SMS handlers could be an example of internal clients that run
>    inside the CORE (in their own "SMS-processor" component).
>
> In addition to hosting and publishing the components themselves, the
> container also manages a set of shared libraries that components can use to
> provide their services.
>
> a component
>
> Component consists of:
>
>    - a component type - a public interface (stored in artifact) through
>    witch the internal functionality can be accessed
>    - one or more component implementations
>    - When a component implementation is deployed, the deployer can
>    override any of the component runtime parameters.
>       - component implementation + runtime parameters = component
>       deployment
>    - When component deployment is instantiated by the container (loaded
>    into memory) it becomes a component instance.
>       - If allowed by the component implementation, the container
>       may choose to pool more than one component instances.
>
> There can be a set of well-known component types that the container can
> use internally.
>

Yep; the XBean kernel takes care of all of this pretty much; allowing
components to be hot-redeployed as well as having sophisticated
cross-component dependencies etc

[snip]

>
> Only externally available components can be remoted inside the cluster.
> The decision to execute a method call on a local node or a remote node could
> be made at runtime: the container could use the following criteria:
>
> if ([number of requests in queue on local node] * [average time of processing a single
request on local node] >
>     [number of requests in queue on remote node] * [average time of processing a single
request on remote node]
>     + [overhead incurred by remoting])
> then send that method call for execution on remote node.
>
>
FWIW if using, say, ActiveMQ as the remoting/distribution layer - you can
weight local services higher so where possible purely in JVM based
invocations are performed without suffering the overhead of marshalling &
networking etc


This decision logic implies that all nodes have more-or-less up-to-date data
> available about the remote queue sizes, execution times and remoting
> overheads. Some of this data can be updated from the method calls
> themselves, but most of it probably has to be distributed on regular basis (
> e.g every 500 ms)
>
Yeah; you could monitor queue depths via JMX and then use that metadata to
spin up new services etc
http://activemq.org/JMX

> Implementation note:
>
>    - when doing an upgrade the container should compare the old set of
>    default values against the new set of default values. If default values have
>    changed between versions, notify the deployer and ask for course of action:
>    override the new default value with old default value or accept the new
>    default value.
>    -
>
>    would make sense to save configuration settings as XML or as
>    JXPath:value mappings
>
>
Yeah. FWIW yesterday I checked in a spike to XBean allowing the use of JAXB2
to be used to manage the configuration and persistence of POJOs; so you can
boot up an XML, configure the POJOs through a console (such as JMX) then
save the XML again.

>
> configuration
>
> All component deployments are configured with XML configuration files - no
> database is needed + it is much easier to evolve XML schemas and to
> implement versioning.
>

FWIW for large deployments you can end up with a gazillion XML files all
over the place - often changing only slightly for different
locations/machines. So I'd like an option of using LDAP as an alternative to
XML to make it easy to use inheritence & overloading properties for large
deployments.

--

James
-------
http://radio.weblogs.com/0112098/

Mime
View raw message