ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rob Oxspring" <>
Subject RE: Smarter Javac
Date Tue, 20 Nov 2001 00:52:33 GMT

> -----Original Message-----
> From: Peter Donald []
> Sent: Monday, November 19, 2001 10:33 PM
> To: Ant Developers List
> Subject: Re: Smarter Javac
> On Tue, 20 Nov 2001 02:45, Magesh Umasankar wrote:
> > > The problem is that we need to cache the information on a per 
> file basis.
> >
> > Not necessarily - One should identify the arguments that truly 
> change the
> > color of the command - for example, let me use a task with 
> 'CacheId' Foo,
> > during its first run did something equivalent to:
> I think you missed my point ;)  You have to cache information per-file 
> because for some reason you may always compile file X with 
> debugging, while Y 
> with no debugging. This does not mean you can't aggregate the 
> files during 
> compiler execution
> javac -g
> etc

How would you go about this in Ant? Surely in current ant this would have to be split into
two invocations of the javac task? In which case, the colour need only be cached on a per
task basis. The issue of cache size / read speed should be mute here, since a single aggregated
cache file would have a size roughly proportional to the build file.  

In short: The same colour applies to all files processed by a given task invocation, therefore
the colour is defined purely by the attributes (+ inner elements) of the task.

An ant file build.xml:
<project name="xyz" basedir=".">
	<target name="init"><property file="defaults.props"/></target>
	<target name="compile" depends="init">
		<javac debug="${debugmode}" src="..."/>		
		<javac debug="${debugmode}" src="...">	
			<fileset .../>			
			<fileset .../>

Would just have a file build.color:
<project color="MD5-012335765342">
	<target name="init" color="MD5-012335765342">
		<property color="MD5-012335765342"></target>
	<target name="compile" color="MD5-1231345345">
		<javac color="12341234"/>
		<jaavc color="57473982"/>

This way you could also regard the configuration colour to have changed even when the only
thing to have changed is the depends attribute of the containing target.  (Can't quite think
whether this is useful or not though - its late)

If the colour is calculated at the getting and setting stage (i.e. only the attributes explicitly
set in the XML file) then the colouring would be immediately available to all tasks.  The
tasks themselves could be changed to take advantage of this new information as necessary.

Please shout me down if I'm being daft and missing points here.

Thanks for your time,

> > Note here, that I am not trying to solve dependency checking.  So when a
> > check is made to see if the coloring has changed, it sets a flag which
> > states whether all files are to be recompiled or whether 
> whatever used to
> > happen earlier should keep happening.  The reason why source files don't
> > matter is that the task, as it is currently written, is anway going to
> > process it.
> That I disagree with ;) There is sometimes reasons to have different 
> colorings for different sets of files. For instance you may need 
> to use a 1.1 
> compiler in some parts of the build and 1.2+ in other parts. 
> Alternatively 
> you may need to always compile some parts with debug enabled 
> (think testing 
> code) while other bits may or may not have debug enabled.

Again - compiling different parts with different compilers would surely need different task
elements within the xml?

> > > This can a significant overhead to the compilation step and the exact
> >
> > Not necessarily.  You load the command cache when ant starts up.  
> this is the expensive step. stat() + read() ing a file can take a 
> bit of time 
> on some filesystems. For instance many make build systems will 
> generate one 
> foo.dep file for every foo.c file. This is okay under unix based 
> filesystems 
> up to about 400-700 files (depending on specific filesystem). However in 
> larger systems or if you have the pain of also compiling under 
> win32 FAT fs 
> then you really need to be able to aggregate these dependency 
> files. These 
> aggregated file systems take a lot more to update (ie change one 
> coloring and 
> you will need to write out whole file again) but much faster to read. 
> So it is a basic tradeoff - the bigger the project, the slower the 
> filesystem, the faster you want your reads to be. However for 
> small projects 
> it is the writing step that takes the most time.
> > > I guess we need to identify the types of "dependency" information. The
> > > two that we identified last time where
> > > 1. structural
> > > 2. coloring
> >
> > Nice terms ;-)
> >
> > But they don't have to be tightly coupled. 
> agreed.
> > Coloring refers to the state of
> > the 'command' while structural refers to the state of the 
> fileset that the
> > command works with.  One can exist with or without the other?
> yep - but as I would contend that the coloring needs to be 
> maintained on a 
> per file basis it does not cost a huge amount to add the structural 
> dependencies in at the same place (Even if they are null atm).
> > > Now how to store this information? There is two extremes. One extreme
> > > saids use a single cache for whole project while the other 
> extreme saids
> > > have a single dependency cache file per source file. There are also
> > > inbetween styles (ie cache per package or perhaps per hierarchy).
> >
> > If we store the cache at a task level and assuming not all 
> tasks are going
> > to support Colorable, we will have only a few entries in the cache on a
> > per project basis.  But...
> I think the dependency data should be done on a per task basis 
> and each task 
> gets to decide how to handle it. We could set up a default 
> directory to cache 
> dependency information (think magic attributes like the classpath 
> stuff for 
> gump) ... not sure.
> >
> > > However we need to leave it up to the build writer how to store this
> > > cache
> >
> > ...the user may specify it as well?
> yep. It is only really necessary when the projects are medium to 
> HUGE. It is 
> something that Build Engineers should be able to twiddle for specific 
> environments.
> > > Unfortunately that only works if all the files were compiled
> > > successfully. If not you have to figure out which ones were 
> updated and
> > > which failed and update dependency scanner/cache appropriately.
> >
> > This would become mute if we throw the files out of the picture.  But
> > again, my fundamental idea of not considering files may be dead 
> wrong. ;-) 
> > So, please provide your views.
> Well I guess we could separate it out into multiple 
> layers/stages. I would 
> still like the dependency cache file have "room" for the structural 
> dependencies though.
> > > However this level of sophistication may have to wait till 
> ant2 because
> > > it is a lot of work.
> > >
> > :-(
> That is of course unless you want to do the work (and there is a 
> damn lot of 
> it - not an easy bit of coding IMHO), while still being backward 
> compatible. 
> If so ... woohooo ! ;)
> -- 
> Cheers,
> Pete
> *-----------------------------------------------------*
> | Never argue with an idiot, they'll drag you down to |
> | their level, and beat you with experience           |
> *-----------------------------------------------------*
> --
> To unsubscribe, e-mail:   <>
> For additional commands, e-mail: <>

Do You Yahoo!?
Get your free address at

To unsubscribe, e-mail:   <>
For additional commands, e-mail: <>

View raw message