ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deryk Sinotte <deryk.sino...@audesi.com>
Subject Requirements for Ant 2.0
Date Wed, 21 Mar 2001 21:19:52 GMT
Sorry about not putting the proper prefix on the subject but I wasn't sure
what it should be.

The following is a fairly lengthy overview of requirements that we've found
would be useful additions or fix existing issues with Ant.


Design suggestions for Ant 2.0
==============================

Prepared by Wind River Systems Inc.
author: Julian Bromwich
email: julian.bromwich@windriver.com

Submitted for March 21, 2001


Contents
========

- Summary

- Fundamental design changes to make Ant more powerful:
   1) Scoping is definitely the biggest problem I can see.
   2) Targets need to be considered more as "methods".
   3) Flow control is the second biggest problem after scoping.
   4) Data types need to be more powerful.
   5) XInclude needs to be implemented.
   6) Class-level dependency checking.
   7) The Ant documentation needs to be better.
   8) Nested elements...

- More standard tasks


=======================================================================

Summary
-------

  Ant's original goal was to be a platform independent "make" utility.
  It has far surpassed this goal and hints of being a very powerful,
  extensible language.  It does, however, have several shortcomings
  which, at present, prohibit it from reaching this state.  I have
  prototyped some fundamental changes to Ant to address some issues.
  I cover these changes as well as some changes that I would like to
  see below.
  
  Many of the problems with Ant are most visible in the scenarios
  where one build file calls another (potentially identical) build
  file, or recursively calls itself.  Scoping is the biggest problem
  here, and I will go into that more later.
  
  Also, many problems I encountered in creating my own Ant tasks
  centered around the fact that many components of Ant weren't made to
  be easily extended.  Many things were private that should probably
  have been protected or even public.  In contrast, some things such
  as the "properties" and "userproperties" hashtables were essentially
  public and really should have just had accessor methods to retrieve
  individual entries.
  
  In Ant 2.0, the architecture needs to be carefully designed to allow
  maximum flexibility to the Task writer while still properly
  protecting access to core components.
  
  If you would like any of the source code to the extensions that we
  have done, please just let me know and I will send them to you.
  
  Good luck in your Ant 2.0 design cycle.  I will try to participate
  as much as I can... if I am given time.  :)


Fundamental design changes to make Ant more powerful
----------------------------------------------------

1) Scoping is definitely the biggest problem I can see.

   Scoping currently is the cause for all kinds of un-expected
   behaviour.  The one which comes to mind is demonstrated in this
   following over-simplified example:

   <target name="myself" unless="base_case" >
     <testbasecase property="base_case" />
     <antcall target="myself" />
   </target>

   Quite obviously, this is intended to recurse until the base-case is
   true.  As it stands, you can't really do this in Ant for a several
   reasons:

    1) Child call always inherit the parent's variables
    2) The child can't re-assign those variables
    3) "if/unless" only check for the existence of a variable,
       not its value.

   For now, I can get around this problem with my "variable" task
   which allows me to re-assign or completely destroy a property...
   a hack indeed. :)

   What is really needed is scoped data.

   - A build file should be considered much like a class.
   - A target should be considered more as a method.
   - Local variables (within a target) should be distinguished from
     globals.

   This implies the following scoping rules:
   - A target can see it's local variables and the globals within the
     current build file.
   - When one target calls another, the called target only has access to
     the globals that are within its build file, not the globals of the
     calling target's build file.

   The "property" and "UserProperty" hashtables are just weird.  Who's
   idea was that!?  That needs to be re-thought.

   Perhaps think about structuring the data space more in the way that
   a language would with a heap for each project and a stack for each
   target invocation.

   The scoping problem doesn't look to me like it will be easily
   solved, but I think it is very important to spend the time to come
   up with a good solution now that won't have to be changed in the
   future.


2) Targets need to be considered more as "methods".

   - The ability to return values to the caller is needed.
     Suggested syntax:

       Calling target:
         <invoke antfile="some_build_file.xml" target="some_target"
                 property="my_return_val" />

       Called target:
         <return> value="some string" </return>

     "Invoke" would replace both the "Ant" and "AntCall" tasks since
     the "antfile" parameter would default to the current build file
     if not specified.

     Upon returning from calling "invoke", in this case, the property
     "my_return_val" in the local scope would have the value "some
     string", or would not be set if the called target did not return
     a value, or if the "property" parameter was not specified in the
     calling "invoke" task.

     Again, this is something that I have prototyped and have been
     using with good results for a while now.  I also consider this
     feature to be vital.


3) Flow control is the second biggest problem after scoping.

   - There need to be conditionals.
   - A target should also have the option of a return statement for
     early abort.
   - Conditionals would be nice too.  There are a couple incompatible
     options here:
       1) <and> and <or> tags
       2) <if test="(A | B) & ~C" > where a standard expression syntax
          is used.

     (I prefer the second option myself, since there are already
      standards and supporting libraries out there for this.)

     Maybe something like this just off the top of my head, or maybe
     just use one of the many existing standards.  XSLT does something
     like this and would be worth a look.

     <target name="example">

        <if test="some_test">
          <echo message="early return" />
          <return/>
        </if>

        <if test="(A | B) & ~C" >
          <echo message="normal return" />
          <return value="2" />
        </if>

        <echo message="end of target" />

     </target>

   - There are several standards for how to implement conditionals
     in XML.
     The XSLT spec is at http://www.w3.org/Style/XSL/
     It allows for conditionals such as <if> and case statements:
     <choose> <when> <otherwise> with the use of a test="conditional"
     syntax.


4) Data types need to be more powerful.

   This can only be properly accomplished if the "scoping" issue above
   is first addressed.

   I have added a "variable" and a "list" datatype which I consider to
   be absolutely necessary for Ant to be powerful.  The "list"
   datatype opens up whole new areas for Task creation and has been
   very useful.

   I had the situation where I needed to create a software load from
   several components.  Each component compiled in exactly the same
   way but had different dependencies on other components.  To compile
   component "A", it was most convenient to create a task to determine
   the list of all the components that needed to be compiled first.
   By calling each component's "build.xml" file (which returns a
   classpath as a "list"), I was able to dynamically build the
   classpath that component "A" needed to compile.  This was most
   often a multi-layered process and the "list" datatype allowed the
   returning of an extensive classpath list back through several Ant
   call-stacks.  At the top of the stack, it is eventually converted
   from a "list" (which is a Vector internally) to a path string via
   the list's "tostring" parameter.  The reason I kept the classpath
   as seperate elements in a list rather than combined into a single
   colon-separated path, was that I needed to easily be able to check
   that I was not adding repeat paths into the list. (see
   unique="true" below).

   Examples:

    <!-- Overwrite the previous value of language with a new one -->
    <variable name="language" value="english" />

    <!-- Destroy the skip_clean variable (a work-around hack) -->
    <variable name="skip_clean" remove="true" />

    The "remove" parameter destroys the variable (removes it from the
    hashtable), but this is a hack to get around the fact that the Ant
    "if" cannot check for anything other than the existence of a
    property.  It also allows me to get around the scoping problems.
    Normally, once I set "skip_clean", all targets that are called in
    other build files also have "skip_clean" set by inheritance, and
    this is not the desired behaviour.

    <list create="jar_list" />
    <list name="${jar_list}" addlists="${invoke_return_list}"
          unique="true" tostring="path" property="classpath" />
    <list name="circular_dependencies" add="${thisfile}" unique="true"/>
    <list name="${import_list}" containslist="circular_dependencies"
          property="circular_error" />
    <list name="circular_dependencies" remove="${thisfile}" />
    <list name="${jar_list}" add="${bundlejar}" />

    The "create" functionality is another hack to get around the lack
    of scoping in Ant.  It just returns a unique name since the "list"
    dataspace is global and there would be a collision if the script
    was recursive.

    Other than XInclude, I have found the "list" datatype to be the
    most useful thing I have added to Ant's repertoire.

    It also facilitates many other useful tasks such as these examples:

    <!-- Set skip_clean if ${myjar} is more recent than everything in
         the ${dependency_list} -->
    <uptodatelist property="skip_clean" targetfile="${myjar}"
                  list="${dependency_list}" />

    <!-- find all build files within the BUNDLE_PATH, and return a
         list of them in "buildfile_list" -->
    <find path="${BUNDLE_PATH}" name="build.xml" prune="true"
          property="buildfile_list" />

    <!-- Invoke build on all dependencies and put their return values
         in "classpath_list" -->
    <invokelist list="${import_list}" target="${buildtarget}"
                property="classpath_list" />

    <!-- To convert the list "classpath_list" into a path string called
         "classpath" do this... -->
    <list name="classpath_list" tostring="path" property="classpath" />


5) XInclude needs to be implemented.
   
   I have implemented XInclude for Ant, by allowing Ant to classload
   and run a specified pre-processor (-Dpp="Xincluder"), but I think
   Xinclude should be directly supported by Ant.

   As an example of what this looks like, here's a snipit of a build
   file which includes the definitions for some of the tasks that I
   have written:

   <!-- Wind River Ant extensions -->
   <xinclude:include href="file:/${WIND_HOME}/ant/windtasks/windtasks.xml"
                     parse="xml" >
     <xinclude:children />
   </xinclude:include>

   The above block is replaced by Ant's pre-processor with all
   thechild elements of the xml file called windtasks.xml.  Only the
   <taskdef> elements are included since the xinclude specifies
   <xinclude:children />.

   Here's a shortened version of windtasks.xml (which is being
   included above).

     <?xml version="1.0"?> 
     
     <antTaskDefinitions>
       <taskdef name="abort"
                classname="com.windriver.antx.taskdefs.Abort" />
       <taskdef name="canonical"
                classname="com.windriver.antx.taskdefs.Canonical" />
       <taskdef name="getlocalhost"
                classname="com.windriver.antx.taskdefs.GetLocalHost" />
       <taskdef name="httpserver"
                classname="com.windriver.antx.taskdefs.HTTPServer" />
       <taskdef name="invoke"
                classname="org.apache.tools.ant.taskdefs.Invoke" />
       <taskdef name="invokelist"
                classname="org.apache.tools.ant.taskdefs.InvokeList" />
       <taskdef name="list"
                classname="com.windriver.antx.taskdefs.List" />
       <taskdef name="uptodatelist"
                classname="com.windriver.antx.taskdefs.UpToDateList" />
       <taskdef name="variable"
                classname="com.windriver.antx.taskdefs.Variable" />
       <taskdef name="xslt"
                classname="com.windriver.antx.taskdefs.Xslt" />
     </antTaskDefinitions>

   Here are some of the uses that I have found for XInclude:

     - The "build.xml" for a particular project can "extend" a base
       "project.xml".

     - By specifying targets in "build.xml" of the same name as those
       in "project.xml", before the Xinclude statement, I can override
       particular targets from their default.  (This is SO USEFUL
       since we compile many "nearly-identical" components that all
       share the same complicated build process, but often need to
       customize the process slighty.)

     - (as above) Easily include all taskdefs for my Ant tasks.

    XInclude is another standard from w3.org, though I have extended
    this standard somewhat to allow for partial includes of an XML
    tree.  Ant 2.0 should probably support the XInclude standard as
    described w3c (once they get it finalized) and then perhaps I will
    try and get my extension supported as part of the w3c XInclude
    standard.

    Our XInclude syntax also allows for the following include which
    specifies only parts of an XML file to be included:

    The original source file:
    -------------------------

      <?xml version="1.0" encoding="iso-8859-1"?>

      <project xmlns:xinclude="http://www.w3.org/1999/XML/xinclude">

        <xinclude:include href="file:/${WIND_HOME}/include/header.xml"
                          parse="xml">
         <A name="2"/>

         <A name="1">
           <A1/>
         </A>

         <B>
           <xinclude:children/>
         </B>
       </xinclude:include>

     </project>   


   The file to include ("header.xml"):
   -----------------------------------

     <?xml version="1.0" encoding="iso-8859-1"?>
     
     <childmaintag>
     
       <A name="1">
         <property name="1A" value="1" />
         <property name="1B" value="2" />
         <A1>
           <property name="1C" value="3" />
         </A1>
       </A>
     
       <A name="2" bob="3">
         <property name="2A" value="1" />
         <property name="2B" value="2" />
         <A1>
           <property name="2C" value="3" />
         </A1>
       </A>
     
       <B name="1">
         <property name="B1" value="1" />
         <property name="B2" value="2" />
         <B1>
           <property name="B3" value="3" />
         </B1>
       </B>
     
     </childmaintag>
          

   The original source file after the XInclude is processed:
   ---------------------------------------------------------

     <?xml version="1.0" encoding="iso-8859-1"?>

     <project>

       <A name="2" bob="3">
         <property name="2A" value="1" />
         <property name="2B" value="2" />
         <A1>
           <property name="2C" value="3" />
         </A1>
       </A>

       <A1>
         <property name="1C" value="3" />
       </A1>

       <property name="B1" value="1" />
       <property name="B2" value="2" />
       <B1>
         <property name="B3" value="3" />
       </B1>

     </project>   


6) Class-level dependency checking.

   This needs to be addressed.  We haven't explored this enough yet to
   make too many suggestions on how this is accomplished.  Our
   developers have asked me several times about this, but I don't know
   how to easily provide it in a compiler-independent way.


7) The Ant documentation needs to be better.

   - Subtle behaviours of Ant are not properly documented.  For
     instance, it does not say (see the example below) which runs
     first A or B, or if "A" runs at all if "B" fails, etc...  I had
     to experiment quite a bit to determine how it behaves, since I
     was setting the value of "B" in the task "A".  There just needs
     to be a bit more explanation in the docs on how some of this
     "core-Ant" stuff works.

       <target name="example" depends="A" if="B" unless="C" >
       </target>

   - There also need to be more examples, especially showing how
     nested elements work.

8) Nested elements...

   - As it stands, nested elements only work if the task supports that
     nested element specifically, but how would we get this example to
     work?

     <if test="condition>
        <some_arbitrary_block>
     </if>

     The <if> task (if indeed it is ever made) would not know about
     all tasks which might be within it.  This needs some careful
     thought...  and I haven't given it much yet.  :)

     Nevertheless, I often wish I could do something like the above
     example, and I'll bet others do too.


More standard tasks
-------------------

  Here are some tasks that I implemented and found very useful and may
  warrant including.  (I will provide source code to any of these that
  you wish of course)


  - abort
      System.exit if condition true.

      <abort if="circular_error"
             message="Circular dependency detected in ${thisfile}" />

  - canonical
      Expands path relative to current build file by default, or
      relative to path of original invocation if callerpath="true".
      (This is unbelievably useful when you have a lot of ant files
      calling each other in various directories).

      <canonical name="thisfile" value="${ant.file}" />

  - getlocalhost
      Get IP address of current machine (This can be problematic in
      Java (host with multiple interfaces) so we should make sure that
      this task supports the full capability.).

      <getlocalhost property="ip_address" />

  - httpserver
      Registers files in a small http server.
      (addFile="virtual_name, actual_name")

      <httpserver base="${base_url}"
                  addFile="/${bundlename}, ${bundlejar}" />

  - invoke
      Calls an target (optional) within an antfile (optional) and
      allows for a return value.

      <invoke antfile="${WIND_HOME}/windcore/osgicore/build.xml"
              property="jar_list" />

  - invokelist
      Same as invoke, but invokes a list of antfiles, with a
      particular target.

      <invokelist list="${import_list}"
                  property="invoketarget.return"
                  target="${buildtarget}" />

  - list
      Creates and processes lists.

      <list name="${jar_list}" addlists="${invoke_return_list}"
            unique="true" tostring="path" property="classpath" />

  - uptodatelist
      Just like "uptodate" but on a list (Current "uptodate" task
      should really instead be able to process a list rather than
      create this new task).

      <uptodatelist property="skip_clean"
                    targetfile="${bundlejar}" list="${jar_list}" />

  - variable
      Just like "property" but re-writable.

      <variable name="base_url"
                value="/com/windriver/internal/bundle" />

  - xslt
      Calls xslt (XML Stylesheet translation).

      <xslt input="file:/${manifest_xml}" output="${manifest_mime}"
            xsl="file:/${manifest_stylesheet}" />

  - find
      Like Unix "find" command but starts from multiple base
      directories that are specified in "path".

      <find path="${BUNDLE_PATH}" name="build.xml" prune="true"
            property="buildfile_list" />

Mime
View raw message