ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Vogel <pvo...@arsin.com>
Subject RE: Did somebody say Shut up and Write? :)
Date Wed, 27 Dec 2000 21:10:31 GMT
I'd like to echo this, JDD's proposal "feels right" from my 
12 years perspective doing configuration management/automated
build/deployment.

To draw from a tool that I think deserves to have some ideas
stolen and incorporated into ant, the mergeing of targets from
multiple project files is handled by CONS in the following 
way:

At the top level, there is a "Construct" file.  Invocation
of "cons" in a subdirectory will lead to an upward search
for a file named "Construct".  Subdirectories/subprojects
anywhere contain a "Conscript" file.  This is by convention,
a Construct and a Conscript file are the same, with the 
exception that the initial invocation of "cons" looks for 
Construct, and the Construct file gets command line args passed
to it.  However, once cons has started, a Construct and a Conscript
file are equivalent.

The "Build" command specifies a list of files (not
necessarily, but usually subdirectories) that contain 
conscripts indicating how they are to be
built.  In a simple tree, the Construct file will usually do
this:
	Build qw(
          drivers/display/Conscript
	    drivers/mouse/Conscript
          parser/Conscript
	    utilities/Conscript
	);
Thereby laying out the whole build tree in one location.  A
more complex environment may have one build requiring a component
that is typically maintained separately, but is incorporated
into this larger build.  For example, if the "drivers" set is
normally maintained separately, it might have its own Construct
file, like this:

[File - mysystem/Construct]
	Build qw(
	    drivers/Construct   # prt of mytree, but mnt. seperately
	    parser/Conscript
	    utilities/Conscript
	);

[File - drivers/Construct]
	Build qw(
	    display/Conscript
	    mouse/Conscript
	);

In this way, "mysystem" doesn't need to know anything about 
how "drivers" is built, or even what subcomponents it contains
(i.e. I might add a "joystick" subdirectory of the drivers project,
and mysystem will still work correctly and incorporate the joystick
build automatically).  This is very similar to the "import" concept
others have mentioned on the list.

The beauty of how Cons handles all this is that it first reads all the 
Construct/Conscript files, so the ordering of the Build list does not
affect the behavior of the system, cons will, after reading all conscript 
files, generate a DAG of dependencies both explicitly mentioned and 
automatically scanned.  This DAG is then traversed, verifying that each 
target in turn is up to date.  Up-to-date checks are based not on
timestamps,
but on MD5 signatures based on everything that goes into generating a
derived
object, that includes, for example, the source file, any imported library
files,
the compiler itself, and the command line to the compiler.  This guarantees
that everything that should be rebuilt for a particular build is rebuilt.
It
also means that I can add an entire tree to my build just by modifying my
top-level
Construct file and know that that new tree will get built correctly,
assuming
it too is built using Cons.

This is the sort of thing that I'd like to see Ant move toward, and I think
JDD's
proposal is on the right track for this to happen, with the exception that I
think
a Workspace and a Module are, essentially, the same as Cons' distinction
between
Construct (Workspace) and Conscript (Module), where a module may, in fact,
have
multiple parents, and the parental relationship is determined by the parent,
not
the child.  In other words, working on one project, call it X, module A may
be
a direct descendant of the top level, while in another project, call it Y,
module
A may be a child of another module within project Y.  A third project may
even
incorporate module A twice, built in two different ways (though I suspect
that in
JavaSpace this is rare)!.

Thanks!
-Peter
--
Peter A. Vogel
Manager, Configuration Management
Arsin Corporation
4800 Great America Parkway Suite 425, Santa Clara, CA 95054



> -----Original Message-----
> From: mpfoemme@ThoughtWorks.com [mailto:mpfoemme@ThoughtWorks.com]
> Sent: Tuesday, December 26, 2000 3:41 PM
> To: ant-dev@jakarta.apache.org
> Subject: Re: Did somebody say Shut up and Write? :)
> 
> 
> 
> ok, here are my thoughts/complaints/requirements for ant 2.0. 
> JDD's doc is
> a great start, and I'd like to get my 2 cents in...
> ---
> I know I've said this before (and have submitted a prototype 
> for a possible
> solution), but we need a way to merge targets from multiple 
> project files
> into a single dependency graph. Here are some cases where 
> this might be
> needed:
> 
> 1) The user wants to break down a large antfile in smaller 
> ones, possibly
> in subdirectories, to simplify things (ie to make stuff 
> easier to find and
> to reduce merging).
> 2) The user wants to pull out code that can be shared with 
> another project.
> Both projects should be able to include the subproject as is, without
> modifying its antfile (ie changing its "workspace" attribute).
> 3) The user wants to compile the code from a third party 
> project (xerces,
> for example) as part of building their project. The third 
> party project may
> or may not be in a subdirectory of the main project.
> 3) The user wants to load a custom task from another project, 
> and possibly
> compile the task before loading. Ideally, there won't be two totally
> different mechanisms for this (ie a <taskdef> statement vs an 
> entry in the
> "taskpath").
> 
> The current way to handle these cases is via the "ant" task. 
> However, the
> concept of an "ant" task (or an "execute-target" task, as mentioned in
> JDD's write up) simply doesn't work for large projects, since large
> projects usually consist of graphs (not trees) of 
> subprojects. Ant must be
> able to analyze the *entire* graph of dependencies before 
> building, and
> dependencies hidden in tasks (and not declared using the "depends"
> attribute) prevent this. So your targets basically become 
> functions, and
> the result is that targets get built twice, or not at all, 
> depending on how
> you've set things up (same as recursive make).
> ---
> I realize that a lot of people use the ant task to run the 
> same script with
> different sets of properties, however. This is basically a poor man's
> "template" or "macro" and is not a reason to keep the ant 
> task around - it
> just means that we need to come up with a real macro system. 
> The expansion
> of these macros should be done at parse time so that by the 
> time the build
> phase begins, all of the dependencies can be sorted out. Here are some
> examples that I think need to be addressed:
> 
> 1) The user wants to execute the same set of steps for each 
> subdirectory,
> ie compile all the files, put them in a jar with the same name as the
> directory, copy the jar to "dist/lib". I believe the netbeans 
> project does
> something like this.
> 2) The user wants to change an attribute for all tasks of the 
> same type, ie
> set debug="true" for all javac tasks. Most projects I've seen define a
> "debug" property that must be specified in each "javac" task, 
> and is easy
> to forget.
> 3) The user wants to execute multiple tasks with a simple 
> command. So a
> "compile" tag might get turned into some "preprocess" task 
> followed by a
> "javac" task.
> 
> One option is XSLT. It would be powerful and easy to 
> implement, but the
> learning curve might be a bit too steep...
> ---
> I also have a question about the ability for tasks to access 
> the ant object
> model. One thing that JDD mentions in his doc is that "the 
> entire build
> tree is represented in memory and available to all tasks that 
> run". Is this
> really such a good idea? How many tasks actually take 
> advantage of this?
> 
> I'd prefer to see a much tighter contract between tasks and 
> their invokers.
> The biggest thing ant has going for it is the sheer number of 
> tasks that
> have been defined for it. This api should be as small as 
> possible, so that
> it's easy to use those tasks in a variety of settings, and to 
> future-proof
> those tasks. I'm thinking of something EJB-ish like:
> 
> public interface Task {
>      public void setTaskContext(TaskContext ctx);
>      public void execute() throws BuildException;
> }
> 
> public interface TaskContext {
>      public void info(String msg);
>      public void warn(String msg);
>      public void error(String msg);
> 
>      public String getName();
>      ...
> }
> 
> Once 2.0 ships, adding methods to the TaskContext interface 
> should be very
> difficult and removing a method or changing a signature should be near
> impossible. By limiting which methods a task has access to, 
> we can make
> sure that the tasks won't need to be rewritten with every release.
> Providing access to the entire object model seems like trouble.
> 
> I agree, however, that it's handy for a script to be able to 
> manipulate
> targets and tasks. But I think this should be done before the 
> build phase
> starts (and after parsing), as opposed to doing it from 
> within a task. That
> way the dependencies can be sorted out before the build phase (sense a
> theme?).
> 
> Matt Foemmel
> ThoughtWorks, Inc.
> 
> 
> 
>                                                               
>                                                      
>                     James Duncan                              
>                                                      
>                     Davidson             To:     
> <ant-dev@jakarta.apache.org>                                      
>                     <duncan@x180.        cc:                  
>                                                      
>                     net>                 Subject:     Did 
> somebody say Shut up and Write? :)                       
>                                                               
>                                                      
>                     12/19/2000                                
>                                                      
>                     06:05 AM                                  
>                                                      
>                     Please                                    
>                                                      
>                     respond to                                
>                                                      
>                     ant-dev                                   
>                                                      
>                                                               
>                                                      
>                                                               
>                                                      
> 
> 
> 
> 
> Ok. So, I heard ya. And I've started a write up. If this is 
> an attempt to
> actually build consensus on Ant2 and earn my stripes, so be 
> it.  In any
> case, Jon was correct in his phone call to me that it was 
> time to put up or
> shut up.
> 
> http://www.x180.net/Projects/Build/index.html
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ant-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: ant-dev-help@jakarta.apache.org
> 

Mime
View raw message