cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Fagerstrom <>
Subject [Design] JXTG 2.0
Date Wed, 24 Nov 2004 19:50:07 GMT
Here are some ideas about how to implement JXTG 2.0. I will not have 
time do start doing any implementation work in the next week or so. But 
I write down my ideas this far if somebody else (Leszek or Jonas e.g.) 
feels like starting the work (and above that happen to be interested in 
my design ideas ;).

I'll focus on describing how pre-compilation can be implemented and how 
to connect to Java written executable tags.

After having evaluated Jelly and taglib, I don't think it is worthwhile 
to trying to implement JXTG2 on top of them. It is certainly a good idea 
to reuse ideas and code from JXTG. One could possibly start from JXTG 
and go to JXTG2 through a sequence of refactoring steps, but the 
monsterous size of the implementation of JXTG makes that less atractive, 
at least for me. I would probably start from Jonas taglib as it contains 
some things that we are going to need any way and is a good starting 
point. Anyway it is IMHO a good idea to take a look at it, and see that 
a taglib implementation can be small and lean and does not have to be 
+150kB source code.


I'll continue to describe a possible design of a template generator (TG) 
that uses pre-compilation and has customer tags. This will be done in 
three steps: We start with a trivial pre-compiled template language (TL) 
_without_ expressions ;) In the next step we add an expression language 
to it and we finish by adding executable tags.

A Trivial Pre-Compiled "Template Generator"

This step is not that useful in it self, the result will just be an 
unnesecarilly complicated implementation of the file generator. This is 
for introducing some ideas.

The pre-compiling TG works in two steps: First it compiles the content 
of the input source to a script, and store the script in its cache 
together with an approriate cache key. Then the cached script will be 
used as long as the cache key is valid. It is important that the script 
is thread safe otherwise it will be unsafe to reuse. As store, Cocoon's 
transient store could be used.

In the second step the script is executed and SAX output is generated.

The Trivial Script

The task of the trivial "template generator" is to output its input 
template document as is :) For this task there allready is a feature 
complete script implementation in Cocoon; o.a.c.xml.SAXBuffer.

The SAXBuffer contains a list of SAXBits where each SAXBit is an 
(immutable) representation of a SAX event. The SAXBit interface contains 
the method "send(ContentHandler)". And the SAXBuffer contains an inner 
class implementing SAXBit for each SAX event type.

The SAXBuffer implements XMLConsumer which records the input events as 
list of SAXBits. It also implements XMLizable with the method 
"toSAX(ContentHandler)" that iterates through the recorded SAXBits and 
execute "send(ContentHandler)" on them.

This far compiling a script means sending the input source to a 
SAXBuffer and executing a script means calling the "toSAX" method on the 
SAX buffer.

JXTG contains code that is similar to the SAXBuffer.

A Real Template Generator

The defining property of an XML TL is that you can embed expressions in 
the XML attributes and text. These expression are replaced with the 
result of evaluating the expressions in some context. To add this to our 
TG we need to define the context, get a expression evaluator and handle 
the expression embeding in the script.

Expression Language

It would be best if we could decide to use JXPath everywhere in Cocoon 
;) But as that not is going to happen we need to support several 
expression languages (EL). This is not just a need in JXTG but also in 
CForms, input modules and many other places. So we have a need for 
plugable EL in Cocoon.

Dmitri Plotnikov, the main author of JXPath have written a common API 
for EL, and commited the code to 
Jakarta Commons Sandbox AFAICS 
the project never took off so we shouldn't use it. But we could probably 
steal some good ideas from it, although I think that we should keep it 
much simpler.

ELs often has the possibility to create a compiled expression object 
from the expression string. When that is possible, (and given that the 
compiled expression is thread safe), we can get better performance by 
storing compiled expressions in the script.

Besides plugable EL we also want to be able to plug in a locale 
sensitive and configurable convertor
component for handling the results from the EL.

Now, we don't have to implement everything at once. We can start with 
suporting one EL and refactor to using the plugable EL, when/if someone 
implements it.

Expression Context

The expresion must be evaluated in some context. JXTG have a lot of code 
that package the different objects in the Cocoon object model so that it 
looks the same way as in flowscripts, but from JXPath or JEXL. Carsten 
have factored out the code from JXTG to 
o.a.c.environment.TemplateObjectModelHelper in scratchpad. It is marked 
as "work in progress" and I don't know the current state of it. But IMHO 
we should base JXTG2 on that (and possibly improve it) rather than 
having still another implementation of the script object model.

Embeding Expressions in the Template Language

In the TL we are using expressions like this:

<tag attr1="text ${expr1} text ${expr2}">
  Text ${expr3} more text ${expr4}, even more text.

We can see that it is the SAX startElement and the characters events 
that are affected. During compilation of the TL into a script, we need 
to parse the attribute content and characters content and create a list 
of text elements and compiled expression elements.

During execution of the script the expression context must be available 
when executing the EL parts of the script.

In JXTG this is implemented in the TextEvent and the StartElement 
classes, where both contains list with interspersed expression and text 

If we continue to base our script implementation on the SAXBuffer, we 
need to extend it with new handling of startEvent and characters like in 
JXTG. We need new StartElement and Characters classes that can handle 
expressions like I described above. During script compilation time they 
need a EL factory. During script execution time they need a expression 
context, so their interface must be extended with something like 
"send(ContentHandler, ExpressionContext)" and simillary the script 
(extended SAXBuffer) need a method "toSAX(ContentHandler, 

Adding Executable Tags

There are numerous ways to add executable tags (ET) to a TL. Of those I 
have seen I prefer Jelly's way, and therefore I'll describe how to 
implement something similar. Take a look on the implementation of some 
Jelly tags

and compare them with the corresponding tags in e.g. taglib and decide 
for yourself what you prefer.

For concretenes lets say that we have a for-tag in our TL (example from 

<core:for var="index" begin="0" end="8" step="2">

We have a TagRepository where URI and tag name is mapped to a tag 
object. Tags that are registred in the TagRepository (executable tags) 
gets special treatment. All others are handled as described in the 
previous sections.

When a ET is executed it will have access to:

* the execution context (including the current expression context)
* the content of its attributes
* its body in form of a script
* the current XMLConsumer for output
* its parent ET (this is less important, but is used for implementing 
e.g. choose/when, take a look at the Jelly tags)

Execution Context

The execution context contains the expression context as before but now 
we want the expression context to be implemented as a stack of (variable 
name, value) bindings, searched from top and downwards for a variable 
binding, (see Jonas code e.g. for details). This is needed for creating 
local variables in template "macros" and for handling context info in 
recursive templates. Recursive templates are needed as soon as you have 
recurively defined data structures like the class stuff in CForms.

The execution context gives access to the tag repository so that it is 
possible to add tags and use tags in user defined tags. It might also 
contain access to Cocoon components, and the source resover.

The Executable Tag

The most complicated question in deciding how the ETs should work is if 
we should require them to be thread safe or not. If they are thread safe 
they can be made part of the script and be completely prepared at 
compile time. But OTH they will be harder to write as one cannot use 
member variables for state info.

For the moment I think not requireing thread safety would be enough and 
that we can see more efficient treatment of thread safe tags as a future 
optimization. In Jelly, tags can implement CompilableTag, which give 
them the possiblity to do some work at compile time, I've no idea about 
how it work though.

Anyway the tag will be created at execute time and need access to the 
different data specified above.

Jelly uses the following inteface for tags:

public interface Tag {
    public Tag getParent();
    public void setParent(Tag parent);
    public Script getBody();
    public void setBody(Script body);
    public JellyContext getContext();
    public void setContext(JellyContext context) throws JellyTagException;
    public void doTag(XMLOutput output) throws MissingAttributeException, JellyTagException;
    public void invokeBody(XMLOutput output) throws JellyTagException;

and setter injection for the attributes. Or this is rather the default 
way, Jelly also use  various reflection based tricks and tries to handle 
almost any bean as a tag.

The "for" tag above would be implemented like this:

public class For extends TagSupport {
  String var;
  int begin, end, step;

  public void setBegin(int begin) { this.begin = begin }
  // other setters and getters

  public void doTag(XMLOutput output) throws JellyTagException {
    for (int i = begin; i <= end; i+= step) {
      if (var != null)
        getContext().variables().put(var, new Integer(i));

(I'm using Jelly interfaces, so the example isn't completely consistent 
with other discussion about intefaces in the mail).

I have no strong opinions about if we should user setter injection, 
getAttribute methods in doTag, etc.

Compiling the Script

Now we continue to extend our script implementation so that it can 
contain an ExecutableTag element as well as the previous ones. An 
ExecutableTag object is set up with attributes, the script for its body, 
its parent and a reference to the factory for creating its tag  bean 
during compile time. During execution we need a "send(ContentHandler, 
Context)" method that creates and instantiates the tag object (as 
described above) and then call its doTag method. The script also need a 
"toSAX(ContentHandler, Context)" for executing all of the script.

Now we need a template parser its not enough to feed the input source 
into the SAXBuffer anymore as we are going to create a new script for 
every body of an executable tag. The parser need stacks for keeping 
track of the current tag and the current script, take a look at e.g. 
Jonas TemplateTransformer to get an idea about how such parsing can be 

Implementing the Tags

The idea is that all tags in JXTG are implemented as executable tags in 
the above described framework. Also tags for defining new tags e.g. macros.

Besides implementing the JXTG tags, we would need to implement the ESQL 
tags (or a replacement of them), and porting (if that is needed) the 
CForms tags. After that we would have a replacement for XSP, AFAIU :)


Having written all this I start wondering if I wouldn't have been better 
to start actually implement it ;)

Anyway, thanks for having read this far.



View raw message