ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Donald <dona...@apache.org>
Subject [Proposal] myrmidon
Date Mon, 04 Dec 2000 13:24:51 GMT
Hi,

A while ago I forked of ant to build a job server (think fancy cron
engine). I ended up evolving it to do things it wasn't originally intended
to do and integrated it strongly with something else I was working on
(Avalon project @ java.apache.org). I always intended to send it back when
I had time to clean it up. Unfortunately Duncan announced his intentions to
propose something and I wasn't sure what I was gonna do the same but I
finished thesis and time freed up so I thought - what the hey ;)

Anyway I would like to put in for consideration myrmidon. 

* What is it?

A genericicised version of ant that goes beyond a dependency manager to a
complete task management environment. It allows various tasks to be created
to be executed in environments other than dependency building. For example
I use it for a cron server and it would be trivial to make a
configure/installshield type app using it.

It is also designed with a clear seperation of concerns (SOC) between
different components and uses Inversion of Control (IOC) where appropriate.
(See http://java.apache.org/framework/developer/index.html for a basic
overview of the design pattern). This should enable easy reuse of different
components and replacement of appropriate parts trivially. 

For instance it should be simple to change it so that tasks were read from
a DB or LDAP server or from encoded binary stream (if writing an
installshield type app). It should also be trivial to plug in XSLT/CSS
front-ends for whatever you need etc. It would also be trivial to plug in
different tasks, data types and conversion rules (for string-todatatype
mapping).

* How is it organized ?

Currently there is 4 different parts to the proposal. The launcher, the
converter api, the tasklet api and the project api. (Eventually there will
also be a data-type registry aswell).

The converter api is basically a registry and a set of basic converters.
With this you can convert between string and complex or primitive types
such as Integers/Fonts/Whatevers. This was designed based on many of the
xml/object serialisation apis such as BML from IBM. It could be expanded to
provide a complete bean serialisation api ala BML but as it stands it is
relatively primitive.

The tasklet api is the API that most people will use and consist of two
parts, engine and tasklets. The tasklets represent each individual task (ie
copy/compile/convert/delete) and is built in a manner similar to servlets
but with a greater SOC. Associated with each tasklet is a context that
gives it information about it's environment (such as current java version,
base-directory of project, task/target/project names, other properties). It
also performs utility functions with that data (currently just resolves
properties and resolves files against project base file).

Each tasklet can also gain access to services offered by the tasklet engine
via  implementing Composer interface and  receiving a ComponentManager
(CM).  From this CM they can gain access to converter registry, data
registry (when implemented), tasklet registry etc. Other services such as
tasklet factories will also eventually be shared via CM so that tasks can
have embedded tasks like has been requested of late.

By accessing everything via the CM and the TaskletContext it allows the
tasklet engine to do a lot more things. For instance there would currently
need to be only 2 changes (basically synchronizing HashMaps) to make the
engine safely parralelizable. It could also scale to having potentially
hundreds of tasks running simultaneously (like in my job server). It also
makes it possible for the services to be easily replaced with different
implementations as long as they implement the required service interface.
So it would be easy to write a stripped down version for an installer if
required.

The project API manages Targets and Projects and is responsible for
building the projects. Again this is designed with a seperation between
interface and implementation so that it would be trivial for a GUI to
generate "wrapper" Project/Target objects that did extra stuff and was
easily editable.

In the future there will be a data registry API that allows new data-types
like patternsets/filesets/filtersets to be dynamically added and used
within build files.

* Features ?

There are a number of features that the structure offers. These are 

1. complete dynamic evaluation of values in every part of build file
2. High reusability
3. High maintainability 
4. Very extensible

By 1. It is meant that the developer of tasks never has to worry about
evaulating values and there is no inconsistencies. Currently there is some
things that will not evaluate ${} inside it (ie includes in filesets) and
it seems to rely too much on programmer. In no cases are properties
exapanded with parsing but are exapanded when the task is run - as has been
requested.

2. occurs due to SOC (and partially IOC). In the case of my cron server I
only use the tasklet and converter APIs. Another product may only use
converter API while the Ant GUI would use all three parts. Maintaining a
clear seperation between different aspects will allow easy reuse between
projects.

3. is basically due to good architecture. It is based on Avalon and
Avalons's Camelot container API which is in turn based on research by the
"smart guys" of the world. You will notice that it shares a lot of
similarities to servlet/EJB and other component based designs and this is
no coincidence as the designers of these architectures presumably were
aware of the research and had access to OOPLSA papers etc. This also has a
downside in that it is difficult to initially grok the ideas there. To that
aim Berin from Avalon project has been slowly building basic documentation
(at link given above). Thou if you are familiar to Aspect Orientated
Programming/Multi-dimensional SOC or other "modern" techniques then the
architecture will feel comfortable to you.

I have attempted to make it 4 by certain programming techniques. Thou in
reality only time will tell if I succeded or not). So at apropriate "bend"
points I have tried to be flexable. For instance at many places there is a
separation between interfaces and implementations. ie you have Target
(interface) and DefaultTarget (default implementation). In the code I may
create the implementation by looking up class based on value of string or
else I will have something like

myService = createService();
...

protected Service createService()
{
  return new DefaultService();
}

This way you can provide your own behaviour via sub-classing (a lil ugly -
but easiest programatically and should suffice for present ant uses).

For those of you who have been hanging around long enough on Ant lists you
may remember when I was clamouring that it would be nice in some cases for
classes to be able to handle their own configurations. Ie so instead of
container setting attributes they get directly handed a DOM-like model and
get to interpret it however they please. Well that has been implemented ;)
This will enable takss to contain other tasks if really desired (thou I
think this is a bad thing). If someone really wants to implement
if/then/else/select/whatever in tasks rather than via xslt then it can also
be done using this (thou I would HATE to see that reach core).

* Products 

Currently there is an extremely bare-bones implementation in CVS under
proposal/myrmidon. To build it just enter that directory and type
build.[bat|sh]. This will by default produce the proposal/myrmidon/dist
directory.  In this directory you will find two directories (bin and lib).
I don't know if the script will work on *nix because I am forced to use
win98 atm but it should be simple enough to modify. For win32 go to
"proposal/myrmidon/dist" and type "bin\ant.bat --help". This will give you
a list of options to use for the program. For a basic run-through type
"bin\ant.bat --log-level=DEBUG -f=../src/make/sample.xmk". This will run
through a basic file.

The tasks and converters for the job will be loaded from
proposal/myrmidon/dist/lib/core.tsk. You will notice that there is
extremely simple meta-data in "TASK-LIB/*.properties" of core.tsk.

If you want to write a task then the best way to do it is to look at Echo
task in org.apache.ant.tasks.core.Echo. You will notice it extends
AbstractTasklet and has a single simple attribute (ie a String "message"
attribute). If you want to create a new task then copy that and maybe add a
few attributes. Just say you want an attribute that has Font as a
parameter. ie

public void setFont( final Font font )
{
 ...
}

Then what you will need to do is also write a converter that converts a
string into a Font. So you may have to parse "Dialog, BOLD, 12" to produce
"new Font( "Dialog", Font.BOLD, 12 );".

After creating your converter and your tasklet then you need to put
meta-info about them into property files (ie
org/apache/ant/convert/core/converter.properties and
org/apache/ant/tasks/core/taskdefs.properties).

Compile and run over suitable build file an whala !

This automagical type conversion will also occur with text data inside
elements. Check out some of the test tasks to get an ide about how to do
this and embedded elements.

* Requirements

Currently the system is 1.2 only - however I don't see this as an issue.
BSDI has recently got a 1.2 JDK which I assume means the FreeBSD jdk is
near to being out of beta ??? Even if it is not - it will be out of beta by
the time that Ant2 comes around.

Currently it is also dependent on CVS version of Avalon but I can organize
a mini-release if that is an issue. It also requires SAX2 atm but this can
be easily changed if needed.

Other than that I think it gives a rough idea of my vision of Ant. It is
still extremely raw (I rewrote it over weekend) and may be missing a few
features (thou I am not aware of any). It also may have extra features (ie
stuff that needs to be removed) because my system was originally aimed at a
croud a lot more sophisticated than target audience for Ant (if you spot
anything like this crept in - inform me and I will nuke it). 

Over next day or so I will add in the basic/core tasks - such a
taskdef/file operations like copy/delete/whatever. 

Future things to do include having a data-types registry, caching of
reflection data (my god reflection is slooooooow) and more sophisticated
validation. By validation I mean that you will be able to specify that task
X needs attribute A and attribute B and attributes C is incompatable with
attribute D etc. This will vastly cut down the amount of code required to
write a task and it will become (hopefully) a piece of cake for tasklet
authors. So your tasks will end up basically removing 95% of checking and
convertion code and just leave the "doing" code.

For those of you wandering what the name means it comes from an ancient
Greek myth. A group of ants were transformed into warriors. The warriors
were of course dedicated and onward marching - and strike fear into
opposition. I thought the name was appropriate ;)

Cheers,

Pete

*-----------------------------------------------------*
| "Faced with the choice between changing one's mind, |
| and proving that there is no need to do so - almost |
| everyone gets busy on the proof."                   |
|              - John Kenneth Galbraith               |
*-----------------------------------------------------*


Mime
View raw message