ant-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Vogel <pvo...@arsin.com>
Subject RE: looping (RE: Foreach task)
Date Mon, 11 Jun 2001 03:42:01 GMT
Hi Jesse,

First, let me thank you for a reasoned reply!  All too often
on this list (which is part of what causes my own tone in many
of my messages here) we have a lot of bickering and not a lot
of *reasoned* debate.

That said, let's see if I can rise to your level! :-)

> 
> You can look at the overall issue as a "teen-parent" problem and act
> accordingly OR you could pose questions that respond directly to the
> opinions committers and non-committers have made.

Well said, so here goes...

> 
> Let me summarize my opinion about the conditional/flow-control issue and I
> hope you will see what I (and some others) are trying to avoid.  
> Since we're using metaphors as well as factual arguments I'll provide a
> more accurate metaphor that describes my goal: I prefer that Ant not 
> implement the Blob Pattern (from the book, "Anti-Patterns", W Brown, 
> Wiley Publishing) by do anything and everything it can, just because it
can.

Great book!  Check out "Patterns and Antipatterns in Software Configuration
Management" sometime too :-)  I agree, Ant should not do anything and
everything just because it can.  It should, however, endeavor to meet
the needs of real projects...

> 
> XML is a data description language.  It was chosen by James for Ant to
> *describe* a build rather than provide process instructions 
> like Make, Jam, etc.  

O.K.  I'll freely admit, I'm late to the XML party, but I think I grok
the basics: xml is a tool to describe data in a standardised manner
in order to serve as a form of "software data bus" -- so, when XML is
used for that (as it is in just about every project I bother to build
with ant :-) it is extremely well suited to the task.

So!  Now lets look at it as the basis for, as you put it, *describing*
a build.  Fundamentally, I can *describe* what needs to happen in order
to build software until I'm blue in the face, so until someone actually
*does* what I'm describing, it's pretty useless, right?  O.K. So here's
where
ant comes in, it reads the description and proceeds to do what is described.
Great!  I think we're all on the same page here.  I remember the first time
I encountered ant, I thought: "wow, what an *elegant* idea!  I was really
excited about the potential (and I still am) for where that idea could go.

Now, let me step back and think about the times I've been asked by 
a developer or my boss to *describe* the build process that I'd architected
(in make), note that I'm using this as an example only, not as a build I
would actually try to shoehorn into ant, ant's a very nice hammer for java,
but it's not well suited to C/C++ based builds.

To begin with, while it is fairly common for a Make-based build process to
require
the developer to painstakingly list all the files that need to be compiled
(or all the object files that are needed, which is essentially the same
thing) my builds were all based on an idea very similar to the "javac" task:
assume that if there's a source file, it's there to be compiled and linked
into a final target library or executable.  The only thing the makefile
needs
to tell me is *what* that target is.  If a source directory is actually used
to build more than one target (say a version of the kernel that expires
after
a given time period for evaluation and another that is for production
releases) 
then the makefile needs to give my process a little more information, like
a list of the target deliverables and which source files need to be 
compiled for each.  Cool, so FOREACH source file, compile it, then link
them all together to form a target, and in fact, my "core" makefiles
(which were included by all the little makefiles) used the GNU foreach
construct to build the OBJS and SRCS variables.

Well! You say, that's what the javac task does! (Indeed) so no foreach
needed,
QED.

Yeah! That's great, you anticipated one *very common* use of foreach.  Now,
what
about this wrinkle: the compiler tree has a bunch of .y files that need to
be 
processed  by yacc to generate the equivalent .c/.h files, and a bunch of 
.l files that need to be processed  by flex to produce equivalent .c/.h
files.

O.K. You say, write a YACC task and a LEX task to loop across your .y and .l
files,
you still don't need <foreach>!

Great, so I anticipated a couple more *fairly common* uses of foreach, and I
hid the foreach in a task.  Now a developer comes to me and tells me that
he's
got a bunch of ".m4" files that he needs processed with m4 to generate a
bunch
of assembly code  which then needs to be assembled.

Well, I could go on FOREACH case where I've used a foreach construct in a 
traditional make-based build system, but I hope you see my point for what it
is:
as the build engineer, I can write tasks out the wazoo to disguise a foreach
in an attempt to anticipate all the cases where a developer is going to want

to loop across a set of things and do something.  But no matter how good I
am 
at anticipating, I'm going to miss some at some point, and in the meantime
I'm
building up a pretty big library of "built-in" tasks that need to be
documented,
etc.

In other words, I'm having to fall into the *very same anti-pattern* which
you
mentioned earlier...

In fact, as you mentioned earlier, the "javac" task *isn't* purely
descriptive -- if I were to *describe* it, I would say this:
	
	
	The JAVAC task looks at EACH .java file, determines class
dependencies,
	orders the list of .java files and, FOREACH file in the ordered
list, calls
	javac to compile it.

In other words javac ISN'T descriptive, but FOREACH is!

Does that mean javac should be eliminated?  Not IMHO, because javac performs
a very common combination of jobs (and we're all used to it :-)

The question, IMHO, is do we want someone trying to develop a build to know
a
certain "core" set of functionality and compose that functionality to
produce
the desired build functionality (construct this system) while people who
didn't
engineer the build can similarly grok the build by understanding that same
core and reading the build engineer's composition (the build.xml file)?  Or
do
we want the build engineer to solve their problems by writing Java to build
an ever-growing list of tasks that must be documented, and must similarly be
understood by anyone trying to grok a build.xml file?  Which, I ask you, is 
more in keeping with the "ant philosophy" that people keep arguing for?

Suppose my name is Inco P. Tent, recent graduate, and assigned to engineer a
build for "waycoolco.com".  I find ant, and I'm flushed with all my pent-up
Java coding talent, so I want to exercise it while I engineer this build.

Well, our system uses some third party tools that are checked into the
source tree as a series of gzip'ed tar files., so clearly I need 
to loop across all these tarballs and extract the tools + libs that I need
for the build to proceed.  No problem, I decide to write a task.  I'm also
a really clever guy, and I'm fond of JRR Tolkien, so I decide (for lack of 
a better name) to call the task "Mithrandir" because I think it's pretty 
magical to do all this in one little task:

<project name="waycoolco" basedir=".">
    ...

    <target name="init">
        <mithrandir basedir="${thirdparty}"/>
        <tstamp/>
    </target>

    ...
</project>


OK, so this is a little bit extreme, but suppose you are Tal N. Ted, one of
the developers on the waycoolco project.  You go to look at the build to 
figure out what you need to do to add your .jar to the build, and you see
this mysterious "mithrandir" thing, you're not all that familiar with
ant, but you thought you grok'ed the basics...  After spending an hour
or two rooting around, you get a little ticked off when you find the 
optional/mithrandir.java file and see that it just loops across a bunch
of tarballs...  What are you going to think of ant?

Wouldn't you rather have seen this?:

<project name="waycoolco" basedir=".">
    ...

    <target name="init">
        <foreach type="files" 
                 property="tarball" 
	           basedir="${thirdparty} 
                 includes="**/*.tar.gz">
	      <gunzip src="${tarball"/>
            <untar dest="${thirdparty}" src="${tarball}"/>
        </foreach>
    </target>
    ...
</project>

Granted, I'm cheating a little, there's currently no easy way way to 
convert "foo.tar.gz" to "foo.tar" for use in the untar, so I'd really have
to do this as two foreach loops, one to gunzip and one to untar, unless
we provide some property manipulation.  But you get the idea.  One truly
*describes* what is being done.  The other hides what's being done in 
a task which is coded (and hopefully documented) outside of the system.

It is in the very nature of describing a build to use the word "foreach"!
Just
*try* to explain javac, delete, copy, jar, etc. *without* using the words 
"for" and "each".


>Unfortunately, Ant didn't do this well from the start since it
> included tasks such as <copy> and <javac>.  Heck, the idea of 
> tasks pretty much set Ant off on the wrong foot.  Over time, some really 
> good descriptive tasks have come forth like the <ejbjar> task.  It has 
> been my hope that Ant's core would progress and mutate to this more
descriptive 
> nature.  

My argument is that <foreach> IS descriptive, and is an *essential* 
descriptive word for a build!

>Why?
> 
> XML is *not* a scripting language and makes for a poor basis 
> to create a scripting language.  Not only that, well defined scripting 
> langauges already exist; some have open parsers to boot!  If Ant really 
> needs iterative declarations to define its build, then it should use a 
> language better suited than XML.  

No argument here, an ant file is not and should not be a script (nor 
should it contain scripts, IMHO) rather it should describe what needs to 
be done in order to build a tree using a well-understood vocabulary (the
core tasks).   In looking at several of the optional tasks (i.e. the
clearcase
tasks) I have to wonder why anyone bothered -- it's just an <exec> or two 
in disguise!  

On the other hand, there are tasks in the optional list that I would think
belong a part of the core vocabulary of build descriptions, propertyfile,
for 
example, though its name is a poor one (I'd like to see all tasks named
as verbs) it's purpose is something commonly found in a well-engineered
build
where a property file may need to be edited/modified depending on whether
the 
build is a developer's private build or a release build, etc.


>I have been examining Python as a basis for 
> about two
> months mainly because my research has shown that a descriptive build
> language will not properly solve complex build situations.  

It will, if you truly allow the use of *descriptive* terms and the community
can stop its knee-jerk reaction to descriptive terms that also happen to 
occur in scripting languages.

> If it were to do
> so, such a language would require a massive paradigm shift with build
> managers; something that is unlikely to happen.  

ant is already a paradigm shift, as is cons (www.dsmit.com/cons).  The fact
is any new technology takes time to be adopted by the people it is intended
to benefit, that doesn't change the need for it...


> 
> Nonetheless, I try to have opinions and voice them in my 
> hopes that Ant does stay close to the descriptive paradigm.  

Laudable.  Ant as it stands is a truly elegant idea with a lot of potential.
I was quite sad to see JDD (original inventor of Ant) essentially thrown off
this list because he dared to suggest that the path the ant committers were
taking was contrary to his vision of what ant could be...

> My goal is not to avoid complexity.

Great!  But that doesn't seem to be true for many of the committers on this
list, who use the word "complex" the same way an exorcist invokes 
the word "demon".

> It's to avoid doing something just because it can be done.  
> It's to avoid creating an XML-based make as that makes less sense than 
> using Make itself.

I don't see any danger of that...

=
> Okay, I've done the tried-and-true "turn the argument around" 
> tactic;  using
> it more for dramatic effect than anything.  In addition, I hope I have
> provided a good explanation of mine (and I think other's) 
> goals with Ant.

I hope you will see my message in the same light.  I hope I've made my
opinions a little clearer.  I find ant to be a truly elegant idea, falling
far short of its potential because of what I perceive to be a misguided
attempt to keep ant's core "pure" (whatever that means, we already agree
that the most useful tasks *aren't* pure w.r.t. the original design goals
of xml -- but that's common too, a tool being used to great effect in a way
that the designer of that tool never anticipated) which *causes* the very
effect that people seem to fear most -- complexity, room for abuse by
the incompetent, and general obfuscation.

Perhaps the best analogy I can come up with is LISP -- here we have a 
*truly* elegant language in which all manner of things can be expressed
as the result of composing 5 basic functions.  Only 5 words needed to 
express a rich variety of ideas and concepts.  Common combinations of those
5 words are, in turn, pre-defined, resulting in an expansion of the 
vocabulary to a degree, but it remained (until the OO folks got hold of it
:-)
a beautiful, simple language of great expressiveness.

ANT has that same potential in the build engineering arena, if we aren't all
lost in the petty bickering and flat-out flaming which seems to be a common 
occurence on this list -- to a far greater degree than I have seen on any 
other open source project which I have followed..


Cheers,
-Peter
> 
> > -----Original Message-----
> > From: Peter Vogel [mailto:pvogel@arsin.com]
> > Sent: Sunday, June 10, 2001 3:47 AM
> > To: 'ant-user@jakarta.apache.org'
> > Subject: RE: looping (RE: Foreach task)
> > 
> > 
> > Because our parents (oops, I mean "the ant committers") are
> > trying to shelter us from "dangerous ideas".
> > 
> > We're all just a bunch of ignorant children who cannot be trusted
> > to use a tool intelligently...
> > 
> > -Peter
> > 
> > > -----Original Message-----
> > > From: Mike Groezinger [mailto:mike@grozinger.net]
> > > Sent: Friday, June 01, 2001 11:49 PM
> > > To: ant-user@jakarta.apache.org
> > > Subject: Re: looping (RE: Foreach task)
> > > 
> > > 
> > > Hi There,
> > > 
> > > the foreach task is also what I'm looking for!
> > > What I don't understand: Why isn't it included in the CVS 
> > rep. in the
> > > taskdefs.optional package (at least I can't find it)? 
> > > Where can I get the latest and maintained version ?
> > > 
> > > Cheers
> > > Mike
> > > 
> > > 
> > > "Nasgowitz, Mark (MED, GEMS-IT)" schrieb:
> > > > 
> > > > Thanks Chris,
> > > > The foreach task with the truncate option is just what I need!
> > > > The problem is that I have been using ant for build, test 
> > > and publish
> > > > results for C code but am unfamiliar with Java. Is it 
> > > possible to get
> > > > the tasks 'pre-compiled' so they can be dropped in? I plan 
> > > on learning
> > > > Java but time does not permit.
> > > > 
> > > > Thanks for any help.
> > > > Mark
> > > > 
> > > > -----Original Message-----
> > > > From: Christopher Berry [mailto:christopher.berry@etrade.com]
> > > > Sent: Thursday, May 31, 2001 10:34 AM
> > > > To: 'ant-user@jakarta.apache.org'
> > > > Subject: RE: looping (RE: Foreach task)
> > > > 
> > > > Greetings,
> > > > 
> > > > In keeping with the spirit of this marvelous list, I am 
> > > posting back an
> > > > updated version of the optional <foreach> Task. It had a 
> > > bug in it which
> > > > I
> > > > have fixed. In addition, I have also added a 
> > type="truncate", which
> > > > passes
> > > > on only the base filename. (i.e. a file named myfile.tar.gz 
> > > will show up
> > > > as
> > > > myfile in the target)
> > > > 
> > > > Cheers,
> > > > -- Chris
> > > > 
> > > > > -----Original Message-----
> > > > > From: Christopher Berry [mailto:christopher.berry@etrade.com]
> > > > > Sent: Tuesday, May 29, 2001 9:22 PM
> > > > > To: 'ant-user@jakarta.apache.org'
> > > > > Subject: RE: looping
> > > > >
> > > > >
> > > > > This looks like just what the doctor ordered. I'm sorry I
> > > > > missed it on my
> > > > > search of the archives -- I searched on "loop"....
> > > > > Thanks (yet again) Diane.
> > > > > Cheers,
> > > > > -- Chris
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Diane Holt [mailto:holtdl@yahoo.com]
> > > > > > Sent: Tuesday, May 29, 2001 8:28 PM
> > > > > > To: ant-user@jakarta.apache.org
> > > > > > Subject: Re: looping
> > > > > >
> > > > > >
> > > > > > A <foreach> task that may do what you need was submitted
to
> > > > > > the list by
> > > > > > Tim Vernum. You can find the original in the archives at:
> > > > > >   
> http://marc.theaimsgroup.com/?l=ant-dev&m=98803020500754&w=2
> > > > > >
> > > > > > And a follow-up version by Chris Greenlee with a "type"
> > > > > > attribute added
> > > > > > at:
> > > > > >   
> http://marc.theaimsgroup.com/?l=ant-dev&m=98842104504477&w=2
> > > > > >
> > > > > > Diane
> > > > > >
> > > > > > --- Christopher Berry <christopher.berry@etrade.com> wrote:
> > > > > > > Greetings,
> > > > > > >
> > > > > > > I am attempting to accomplish the following: For an 
> > > unknown set of
> > > > > > > TAR.GZ
> > > > > > > files in a directory, loop over these files and 
> unpack each.
> > > > > > >
> > > > > > > I checked the archives and do not see a Ant solution for
> > > > > > this problem.
> > > > > > > Is
> > > > > > > that correct??
> > > > > > >
> > > > > > > Some have suggested that this should be done w/ XSLT. 
> > > Is that "The
> > > > > > > Way"?? If
> > > > > > > so, could someone provide a HowTo for this approach??
> > > > > > >
> > > > > > > Or should I write a Task of my own??
> > > > > > >
> > > > > > > Thanks very much,
> > > > > > > -- Chris
> > > > > > >
> > > > > > > Chris Berry  --- christopher.berry@etrade.com  -- 
> > 512-323-9479
> > > > > > >
> > > > > >
> > > > > >
> > > > > > =====
> > > > > > (holtdl@yahoo.com)
> > > > > >
> > > > > >
> > > > > >
> > > > > > __________________________________________________
> > > > > > Do You Yahoo!?
> > > > > > Get personalized email addresses from Yahoo! Mail - only $35
> > > > > > a year!  http://personal.mail.yahoo.com/
> > > > > >
> > > > >
> > > 
> > 
> 

Mime
View raw message