cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Mazzocchi <stef...@apache.org>
Subject Re: [design] Cocoon Blocks 1.1
Date Sun, 03 Nov 2002 23:25:26 GMT
Stephen McConnell wrote:

>
>
> Stefano Mazzocchi wrote:
>
> Stefanno:
>
> Have read with interest the Blocks 1.1 description. First of all -
> thanks to everyone who contributed to this.  I have a number of notes
> in-line, some of which I am sure will reflect my ignorance concerning
> the Cocoon world/terminolgy.  Throughts are strongly related to
> background with Avalon, experience with Merlin and Fortress and usage of
> the excalibur/meta package.

Stephen, thanks much for this. I think your experience in 
component-oriented paradigm will be very valuable for us here. See my 
comments inline.

>
>
> >
> >  +---------------------------+
> >  | Part 2: technical details |
> >  +---------------------------+
> >
> > Ok. Now that we have described where we want to go, let's describe how.
> >
> > Cocoon Blocks
> > -------------
> >
> > A Cocoon block is a zipped archive, just like JARs and WARs.
> >
> > The suggested extension of a cocoon block is ".cob" (for COcoon Block).
>
>
>
> Another suggestion ... BAR - Block ARchive.
>
> The reason for suggestiong this is that the concept of a JAR/WAR style
> deployment unit is something I've been looking at within the Merlin
> framework.  It seems to me that the notion of a block is something
> usable at level across many different applications and based on the
> requirements and descriptions here - I dion't see any immediate Cocoon
> specifics except for the inclusion of the sitemap and default sitemap
> semantics (more notes on that later).

Well, a COB is more than a BAR. A COB is a cocoon-specific BAR. We need 
this. We need a way for the block to *mount* itself onto a specific URI 
space. Otherwise blocks are just a way to deploy java components and 
their libraries.

>
> >
> > The suggested MIME type is "application/x-cocoon-block".
>
> And following the BAR notion .. "application/x-block"?

I'm not against using a more general MIME type, as long as there is a 
way to *cast* it to COB before deployment. Something like having a 
signature inside the block metadata or something like that.

>
> >
> > A Cocoon Block (COB from now on) includes a directory called
> >
> >  /BLOCK-INF
> >
> > which contains all the block metadata and the resources that must not be
> > directly referentiable from other blocks (for example, jars, classes or
> > file resources made available thru the classloader). The directories
> >
> >  /BLOCK-INF/classes
> >  /BLOCK-INF/jar
> >
> > are used for classes and jar files. [This follows the WAR paradigm]
>
>
>
> For consitency with the Servlet spec (Web Applications/SRV.9.5 Directory
> Structure) - I suggest /BLOCK-INF/jar be changed to /BLOCK-INF/lib

Yes, yes, I overlooked that. Totally agree it should be /BLOCK-INF/lib

>
> >
> > The main COB descriptor file is found at
> >
> >  /BLOCK-INF/block.xml
> >
> > This file contains markup with a cob-specific namespace and will
> > include the following information:
> >
> >  1) block implementation metadata:
> >      - unique URI identifier [this identifier will also be used as an
> > address on where to locate the block and how to download it from the
> > web!] (example: http://mystuff.org/dist/myblock-1.5.34.cob)
> >      - version (1.5.34)
> >      - short name (My Block)
> >      - description
> >      - author
> >      - URI of license (http://mystuff.org/dist/license)
> >      - URI of the distribution location
> > (http://mystuff/dist/latest/myblock.cob)
> >      - ???
> >
> >  2) role(s):
> >      the URI(s) of the behavioral role(s) this block implements
> >      and exposes [optional]
>
>
>
> When you are using the work "role", is it safe to assume that this is a
> URI that resolves to a description of the set of computational
> "service"(s) that a block is capable of providing?

yes, it's the 'behavioral contract'.

>  If this is correct - then I would suggest renaming this to 
> "service(s)".  

'service' is kinda abused as a term, expecially now with the web 
services stuff floating around.

> My rationale here is that a "role" (to me) is more correctly aligned 
> with the consumer of a service - Block B1 is depedendent on service X 
> for role of "authorization".  If I understand correctly, the notion 
> you desribing is collection of an interface + version range supported 
> by a block that would enable it to be supplied to Block B1 in order to 
> fulfill the service dependecies that B1 has with respect to its 
> "authorization" concerns.

Your assumptions are correct and I agree with your rationale that 'role' 
is kinda misplaced as an identifier for a behavioral contract.

Still, I don't like the term 'service'. Anybody else has a good 
suggestion for this?

>
>
> Keep in mind that I'm biased relative to the Merlin/Phoenix coventions
> here of using the work "service" to describe the functionality 
> exported by a component. 

No problem.

> I'm extending that notion on the assumption that a block exposed a set 
> (or sub-set) of the services provided by the components it is aggregating.

Yes, this plus all the cocoon-specific services. In fact, you could 
think at the sitemap as a description of the behavior of a hidden 
'cocoon component' that exhibits pipeline-handling services.

> Just as a side note, you may want to think about seperating "block" 
> URIs from "service" URIs.  

Read again: it's already there. One URI describes the block another 
describes the 'behavior' that it's implementing.

So, we could have

  http://apache.org/cocoon/blocks/instances/FOP/3.2 implements 
http://apache.org/cocoon/blocks/FO2PDF/1.2

> This is something I've been working on recently in
> Merlin - and the seperation of component provider for service has 
> proved valuable.  

Oh, it's even more than valuable: it's vital. Otherwise, how can be 
implement block polymorphism?

> It ensures that the concepts of a service is not tied to a
> particular implementation unit (block or component).  Seperation of
> component implemetation meta data from the service meta data is 
> already in place under the excalibur/meta package for the same reasons.
>
> >
> >  3) dependencies:
> >      the URI(s) of the behavioral roles this block expects,
> >      along with the prefixes used by the block as shortcuts in protocol
> >      resolving (see below for the meaning of this) [optional]
>
>
>
> I'm guessing that your referring to the inclusion of a roles file - is
> that correct? 

Nop. I'm talking about the dependencies of a particular block on the 
services provided by other blocks.

So

  http//apache.org/cocoon/blocks/instances/Forrest/1.0 requires 
http://apache.org/cocoon/blocks/FO2PDF/1.[2-x]

which means that Forrest requires a service to transform FO into PDF and 
will use a contract defined on version 1.2 of that behavioral contract 
and not changed until 2.0 (excluded)

> How does this compare to something like the dependencies
> declaration used in the excalibur/meta package?
> http://jakarta.apache.org/avalon/excalibur/meta/dependencies.html
>
> >
> >  4) inheritance:
> >      the URI of the block extended. [optional]
>
>
>
> It seems to me that there two distinct inheritance concerns: (a) block
> inheritance and (b) component inheritance (assuming that a block
> aggregates components).  In the case of block inheritance this would
> handles the cases of resources and the ability to redefine resources 
> in derived blocks.  

My proposal didn't include a way for a block to access a resource 
included in another block directly, but only passing thru a sitemap and 
invoquing pipelines.

Here, when a sitemap *extends* another one, it's means of falling back: 
the two sitemaps are appended and if no matching happens in the first 
one, it falls back on the extended one.

> In the case of component inheritance, this should be
> handled at the component type level and should not be linked to block
> inheritance.

I thought that classloading precedence would solve this issue almost 
automatically. In fact, again, if the asked component is not present in 
the block classloader, it will fall back to the classloader of the 
extended block.

>
>
> >
> >  5) sitemap:
> >      the location inside the block file space of the sitemap
> >      [optional, if not found defaults to '/sitemap.xmap']
>
>
>
> This one - I'm not sure about - does it make sence for this to be part
> of a generic block specification

No, it makes sense to have this in a cocoon block specification but I'd 
be interested in seeing how our effort can combine with others.

> , or is it part of a block that provides
> functionality derived from a stitemap? 

A cocoon block is a block with some cocoon specific service. In this 
sense a COB extends a BAR and for this reason must have cocoon-specific 
semantics.

> Perhaps this is point where a COB extends a BAR ?

Yep. And no small point: without this, blocks are almost useless as a 
webapp deployment tool to us. In avalon, you are packaging services 
provided by components, in cocoon we want to package services but they 
are not only provided by java components but also provided by cocoon 
services (pipelines).

In this sense, if a block exposes a sitemap is a cocoon block, if it 
doesn't (because sitemap exposure is optional) it is a regular avalon block.

> >
> >  6) configurations:
> >      the configurations required for this block to function [optional]
>
>
>
> Some clarification needed here - I'm assuming that a block is a
> collection of a components.  

No. A block is a collection of deployable services, some of which are 
implemented as avalon components.

> Each component would have its own meta info
> (explicit or derived).  At the level of block I can imagine information
> that is describing profiles of component usage, and instructions
> concerning assembly of profiles that will result in the establishment of
> a computation system (I'm talking about internal assembly of a block
> here - not block assembly).  This internal "assembly" level information
> can be considered as the block configuration but should not be confused
> with component configuration data.

I lost you here.

Let me give you an example of what I mean with configuration.

Let me suppose that I deploy a block that provides me with 
authentication services (don't think about java components only, think 
about also a pipeline that handles the login pages, the error pages, the 
authentication flow, the user-managing pages and flow and the components 
to connect to the various data storages)

This block will then need configurations to work such as:

  - system to use [file|RDBMS|LDAP]
  - location of the database
  - username/password for connection (not needed for 'file')
  - ...

and so on.

>
> On the subject of component configuration, there are three different
> levels of component configuration that are handled within the Merlin
> container.  The first type is static configuration defaults (established
> by a developer and bundled with the class), the second type is
> configuration data associated with a named deployment profile (i.e.
> component X deployed using profile P1 is different to component X
> deployed using profile P2).  The third category of configuration data is
> data defined by an administrator that typically suppliments a profile,
> which in turn suppliments default configuration data.

It is *not* the deployer concern to know who uses the configurations 
inside the block. So I shouldn't have to configure single components, 
but the block as a whole and then the block knows what part uses what 
configuration. Otherwise IoC is broken.

I don't want users to have to know the internals of the block in order 
to be able to configure it.

it should be as simple as possible and as transparent as possible to the 
user. Just fill-up the form with the value you want and that's it.

>
>
> >
> >
> > Also, the /BLOCK-INF/ directory contains the 'roles' file for Avalon
> > components:
> >
> >  /BLOCK-INF/roles.xml
>
>
>
> I've been thinking about how to handle roles versus the more formal meta
> data approach used in Merlin. One of the first things that is needed at
> the component level is the declaration of mechanism used to bring
> external data into and meta-data model.  Markus has already started
> working on content in this subject and I'll be shifting some of the
> meta-data content out of Merlin to the excalibur/meta package in the
> near future as part of supporting this work.  In effect there should not
> be a need to include a /BLOCK-INF/roles.xml at the spec level - instead
> one should be declaring a meta management strategy at the component
> level, and possible a default strategy at a block level.  This would
> enable the deployment of ECM style components without change, together
> with non-ECM components.  Specification of the inclusion of a roles file
> would be part a ECM meta strategy spec.

Ok.

>
> >
> >
> > Possible use-case scenario
> > --------------------------
> >
> > Suppose you have your naked cocoon running in your favorite servlet
> > container, and you want to deploy myblock.cob. Here is a possible
> > sequence of actions on an hypotetical web interface on top of Cocoon
> > (a-la Tomcat Manager)
> >
> >  1) upload the myblock.cob to Cocoon
> >
> >  2) Cocoon scans /BLOCK-INF/, reads block.xml and finds out the
> > behaviors this block depends on as well as the block that it extends.
> >
> >  3) the block manager connects to the uber "Cocoon Block Librarian"
> > web service (hosted probably on cocoon.apache.org) and asks for the
> > list of blocks that exhibit that required behavior.
> >
> >  4) the librarian returns a list of those blocks, so the users chooses,
> > or the manager allows the user to deploy its own block that implements
> > the required behavior or to reuse those already deployed blocks that
> > implement the required behaviors.
> >
> >  5) Cocoon checks that all dependencies are met, then unpacks and
> > installs the blocks
> >
> >  6) For each block that exposes a sitemap, the deployment manager asks
> > the deploying user where he/she wants to *mount* that block in the
> > managed URI space or if he/she wants to keep them internal only (thus
> > only available to the other blocks, but not mounted on the public URI
> > space)
>
>
>
> The above comment is probably the point where a COB comes into focusus
> as a specification that extends a more generic BAR specification (i.e.
> COcoon Block could be viewed as an extension of a generic component
> Block ARchive).

Yep

>
> >
> >
> >  7) for each block that requires installation-time configurations, the
> > block manager will present the user information on how to configure
> > the block.
> >
> >  8) If no collisions in the URI spaces are found, the blocks are made
> > available for servicing.
> >
> >
> > Resource dereferencing
> > ----------------------
> >
> > Security concerns aside, the above scenario shows one major issue:
> > blocks are managed, deployed and mounted by the container. There is (and
> > there should not be) a way for a block to directly access another block
> > because this would ruin IoC. xdc
>
>
>
> If you follow the seperation of  "block" from "service" you can avoid
> this issue.  In effect, "service" is what is exposed by the assembly
> system - block never needs to be exposed.  However, this does not
> address the complete picture.  The block concept includes resources as
> well as services.  To complete the picture, the block would need to
> declare accessible resources (something not addressed in the
> excalibur/meta or Merlin system).

You got me wrong here: the separation of block and service was already 
proposed, but for inheritance, you have to expose *directly* the block 
you want to extend. You can't extend a behavior with a block.

Also note that cocoon blocks will not expose resources but only 
pipelines and components. Resources are those generated by the pipelines.

>
> The idea of seperating "block" and "service" has significant
> implications - firstly, the structural unit of deployment are seperate -
> which means that a service interface, realted meta and resources can be
> loaded indepedently of a block.  You need to be able to do this as soon
> as you get into classloader hierachies across which service defintions
> appear higher in the classloader that the implemetations (i.e. the
> service defintions are shared whereas the block implementation is
> protected).

I lost you here again, probably you are more familiar than me on 
implementation details.

>
> >
> > Some design decision taken
> > --------------------------
> >
> > o) NO BEHAVIOR VALIDATION:
> >
> > I thought a lot about it but I think that having 'behavior description
> > languages' (such as the WSDL-equivalent for blocks) is going to be
> > terribly complicated, expensive to implement and hard to use and
> > enforce, even for simple blocks which don't expose a sitemap and are
> > just repositories for informations.
> >
> > For this reason, there is no validation taking place: if a block
> > implements a particular behavior and exposes it thru its descriptor
> > file, Cocoon automatically assume it implements the behavior correctly.
> >
> > In the future, we might think of adding a behavior description layer to
> > enforce a little more validation, but I fear the complexity (for
> > example) of validating stylesheets against a particular required
> > behavior.
> >
> > IMO, only human try/fail and patching will allow interoperability.
>
>
>
> Given sufficient meta-info (type-level) plus meta-data (profile-level)
> it is possible to do  validation on components prior to the assembly 
> of blocks/components into a running system.  

Yeah, for components it's doable. But my concern are is talking about 
validating an entire URI space with its internal flow etc etc. Not easy.

> The validation phase does
> things like ensuring that meta-data in consitent with implemetation,
> references to resoruces actually refer to existing resources, etc.  

Hmmm, suppose you have a matchers like this

  <map:match pattern="news/**">
   <map:call resource="block:newsstore:/news/{1}"/>
   ...
  </map:match>

how are you going to validate it?

> This
> type of validation does not need any supplimentary langauge because its
> simply ensuring the consistency of a logical system before system
> deployment.  Validation could be applied at block creation time, and
> during multi-block assembly.

Components were never my concern for validation purposes. 
Cocoon-specific URI-oriented services are!

>
> >
> >
> > o) VERSIONING AS PART OF THE BEHAVIOR URI
> >
> > The behavior URI *MUST* terminate with a /x.y that indicates the
> > major.minor version of the behavior that a block implements.
>
>
>
> Can you explain the *must* 

I'll explain it like this: Java failed to add versioning to interfaces 
and class definitions (assuming that it was classloading's concern to do 
that). I don't want to make the same mistake here. A contract is 
immutable *only* if marked with a timestamp or a unique version number. 
Unlike W3C, I prefer version numbers as unique discriminators for URIs.

> - the conventions used in the excalibur/meta
> package assume a default value of 1.0 if no version information is
> supplied.  

This is lame. It's like saying that you'll default to 
'org.apache.avalon' if you forgot to indicate the package of a 
component. I think one should know what he/she is doing: if the URI that 
defines a contract is

http://blah/something/1.0

it's not

http://blah/something

> My experience is that this is good for the developer but bad
> for the user. User's typically prefer the most recent stable release 
> as a default value.

Ah, you are mixing concerns here! One thing is to talk about contract 
dependencies, another thing entirely is to talk about user preferences 
of deployment of block *instance* versions!

I agree with you that users want the most stable release, but that's why 
the block descriptor metadata in my proposal includes:

  - unique URI for this block (http://blah/myblock-1.3.239.cob)
  - URI for the latest release  (http://blah/myblock-latest.cob)

but these are URI for block implementations *not* for the block behaviors.

> I've also some reservation about the "/" delimited as the appropriate
> means for version delimiting - because it would break what is already
> running in Merlin :-)

Nothing is carved in stone here so feel free to propose alternative 
syntaxes as long as the intended results remain the same.

>
>
> >
> >
> > On dependencies, each block must be able to specify the 'ranges' of
> > versioning that it is known to work with. For example
> >
> >    prefix="skin"/>
> >
> > But I haven't really thought about the patterns that could be used for
> > this.
> >
> > Please, help on this.
>
>
>
> Some useful documetation concerning "component" level meta info for the
> type level is available on the excalibur/meta package.  This meta info
> *only* deals with the component type level (equivalent to information
> supplimenting the component implementation classes and service interface
> classes).
>  http://jakarta.apache.org/avalon/excalibur/meta/index.html
>
> Meta information concerning the description of "profiles" (the
> configuration data, context directives, etc.) is defined under the
> Merlin 2 API.  The Profile Javadoc is a good starting point.
>
>  http://jakarta.apache.org/avalon/excalibur/merlin/api/

Ok, I'll take a look at those later (I'm offline right now)

>
>
> >
> >  3) Which avalon container should we use since the one we currently
> > use (ECM) is not powerful enough? is there already a container which
> > is powerful enough to handle our needs as described here? if not, what
> > do we do? we implement our own or work with the avalon people to fix
> > theirs to meet our needs?
>
>
>
> I would *very* much like to see this as a joint Cocoon/Avalon iniative. 

Me too. I don't want to reinvent the wheel if it's possible to avoid that!

>
> On the Avalon front there are two containers the play into the
> requirements stated above - Merlin and Fortress.  However, neither of
> these containers completely address the requirements.  But lets look a
> little deeper and figure out where Avalon is today relative to the
> target and what potential is offered by a combination of Merlin,
> Fortress and aother Avalon related iniatives.

ok

>
>
> Defintion of a block as a structural package
> ---------------------------------------------
>
> I would like to see an Excalibur package dealing with a BAR (Block
> ARchive) that serves as the basic structure for a COB.  This should
> include tools and utilities for BAR creation, structural validation,
> signing, etc.  There is existing content in the Phoenix app-server
> related to the SAR file format which is close to the notion of a block
> in terms of structure but is too cause grained for the block concept.

Why so? (BTW, should we copy Peter in this discussion?)

>  In
> addition, the work in Merlin dealing with container defintion seems to
> me to be very close to component/service management side of a block, but
> lacks the formal management of resources (i.e. a Merlin container only
> expose services - not resources).

Again, cocoon blocks should not expose resources so this is not a problem.

>
>
> Component meta info and meta data
> ---------------------------------
>
> As mented above - the component type level meta info in excalibur/meta
> combined with profile level meta data in excalibur/assembly (model
> package) is a working starting point for the component level deployment
> concepts. There is some more seperation work to be done on the Merlin
> side - after which much of the Merlin meta data model will move over to
> the excalibur/meta package.  This will provide a light-weight meta model
> that is container independent.  The model does not currently support
> inhertance - this would require some minor additions to the existing
> stucture and some significant additions to the verification functions.

Ok.

>
>
> Assembly solutions
> ------------------
>
> Merlin includes a assembly engine that automates the process of wiring
> together components based on depedencies and services. This is working
> well today but could do with some refactoring.

Is it working well or not? the above doesn't really parse.

>   Notions of default
> configurations combined with packaged deployment profiles are proving to
> be excellent solutions to simplification of the over service management
> problem.
>
> Lifecycle and Lifestyle management
> ----------------------------------
>
> Both Merlin and Fortress support the classic Avalon lifecycle stages
> (configuration, contextualization, composition/servicing, etc.) together
> with a common model for the introduction of lifecycle extensions.
> Respective implemetations differ in the Merlin allows extension
> implementations to be component that my have their own depedencies
> whereas Fortress does not have support for compoennt assembly. Lifestyle
> management is equivalent in that both provide support for singleton,
> thread, pool and transient policies.  Again, implemetation approaches
> differ - Fortress is very much derived from the ECM model and respects
> lifestyle marker interfaces whereas Merlin requires lifestyle policy to
> be declared within the meta-info of a component type.  Looking forward,
> the Merlin strategy will be to declare the lifestyle processing
> strategy, allowing for defintion of a plug-in handler for lifestyle
> resolution - allowing a mix of pure meta-based components together with
> ECM style and Avalon 4.1.2 marker interface recognition.

So, do you think it would be possible to migrate Cocoon from ECM to 
Merlin/Fortress without breaking existing functionality?

Also, are they capable of run-time changes to the block dependencies?

> Mixed lookup semantics
> ----------------------
>
> Fortress provides complete support for the extended semantics implied
> within a lookup argument.  The Merlin 2 implementation does not support
> this.  The main issue (from my own point of view) is that the Avalon
> framrework Composable and Serviceable interface seamantics are
> insuffiently specificed and the real requirement here is to resolve this
> at the framework level first, then apply these solutions within
> respective containers.  In the meantime, the strategy for Merlin will be
> to plug-in an ECM style manager when required at a component or
> container level (with an implementation based on existing Fortress
> code).  This will enable zero modification of existing ECM style
> components.

Ok, cool. We don't want to deal with framework changes until they 
solidify so this is a very big requirement for us.

>
> >
> >  4) how do we implement the block manager? should it be a comman line
> > interface or a web interface, or both?
>
>
>
> I don't think I agree with the question ;-)
> Management of blocks should be indepedent of the means through
> information is presented. Assume that you have a container that is
> capable of managing a set of components, resources and subsidary
> containers ... you could imagine a management interface to the
> container, and that management interface could be accessible via the
> web, command line, JMX etc.

Yeah, of course. for 'block manager' I was indicating the 
'user-interactive layer' that allows us to deploy, configure and manage 
blocks. I was not talking about the block manager in term of container 
implementation. Terminology conflict here.

>
> > what about security?
>
>
>
> Work I've done in this area is perhaps excessive relative to what you
> have in mind.  I have a micro PKI which handles the generation of keys
> and certificates which are used for both admin and runtime
> authorization. The main difference between the work I'm doing and what
> your describing here is that I'm dealing with distributed containers and
> I need to propergate identify with every invocation and single
> invocation may result in service invocation across multiple container
> deployed in defferent sites, each with different security policies.

No, no, no, nothing that fancy here :) we just would like to be sure 
that the block http://apache.org/cocoon/blah-1.3.398.cob really comes 
from apache.org and is therefore trustfull to deploy. That's plain 
enough for me.

>
> >
> >  5) the 'uber library of cocoon blocks'. Where do we host it? how to
> > we manage it? How do we provide the block discovery web service? which
> > technology do we use: SOAP or REST?
>
>
>
> My experience here is somewhat experimental at this stage.  I'm not
> using a web protocol - instead I'm passing meta model structures over
> the wire (i.e. remote invocations but the scenario is a little diffenent
> because I'm more concerned with service access where service can be
> relocated locally or accessed remotely).
>
> >
> >  6) should we "digitally sign" our blocks?
>
>
>
> Yes.
>
> > if so, how?
> >
>
> Has anyone though about a Cocoon Certification Authority ?

Ok, I think we have to involve our crypto gurus here, I'll ask the 
mod_ssl people.

For now, let's skip this step since it's just another step in the 
validation phase and doesn't impact the block design.

-- 
Stefano Mazzocchi                               <stefano@apache.org>
--------------------------------------------------------------------




---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org


Mime
View raw message