ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Donald <pe...@apache.org>
Subject Re: Smarter Javac
Date Mon, 19 Nov 2001 22:32:56 GMT
On Tue, 20 Nov 2001 02:45, Magesh Umasankar wrote:
> > The problem is that we need to cache the information on a per file basis.
>
> Not necessarily - One should identify the arguments that truly change the
> color of the command - for example, let me use a task with 'CacheId' Foo,
> during its first run did something equivalent to:

I think you missed my point ;)  You have to cache information per-file 
because for some reason you may always compile file X with debugging, while Y 
with no debugging. This does not mean you can't aggregate the files during 
compiler execution

javac -g Bar.java Baz.java

etc

> Note here, that I am not trying to solve dependency checking.  So when a
> check is made to see if the coloring has changed, it sets a flag which
> states whether all files are to be recompiled or whether whatever used to
> happen earlier should keep happening.  The reason why source files don't
> matter is that the task, as it is currently written, is anway going to
> process it.

That I disagree with ;) There is sometimes reasons to have different 
colorings for different sets of files. For instance you may need to use a 1.1 
compiler in some parts of the build and 1.2+ in other parts. Alternatively 
you may need to always compile some parts with debug enabled (think testing 
code) while other bits may or may not have debug enabled.

> > This can a significant overhead to the compilation step and the exact
>
> Not necessarily.  You load the command cache when ant starts up.  

this is the expensive step. stat() + read() ing a file can take a bit of time 
on some filesystems. For instance many make build systems will generate one 
foo.dep file for every foo.c file. This is okay under unix based filesystems 
up to about 400-700 files (depending on specific filesystem). However in 
larger systems or if you have the pain of also compiling under win32 FAT fs 
then you really need to be able to aggregate these dependency files. These 
aggregated file systems take a lot more to update (ie change one coloring and 
you will need to write out whole file again) but much faster to read. 

So it is a basic tradeoff - the bigger the project, the slower the 
filesystem, the faster you want your reads to be. However for small projects 
it is the writing step that takes the most time.

> > I guess we need to identify the types of "dependency" information. The
> > two that we identified last time where
> > 1. structural
> > 2. coloring
>
> Nice terms ;-)
>
> But they don't have to be tightly coupled. 

agreed.

> Coloring refers to the state of
> the 'command' while structural refers to the state of the fileset that the
> command works with.  One can exist with or without the other?

yep - but as I would contend that the coloring needs to be maintained on a 
per file basis it does not cost a huge amount to add the structural 
dependencies in at the same place (Even if they are null atm).

> > Now how to store this information? There is two extremes. One extreme
> > saids use a single cache for whole project while the other extreme saids
> > have a single dependency cache file per source file. There are also
> > inbetween styles (ie cache per package or perhaps per hierarchy).
>
> If we store the cache at a task level and assuming not all tasks are going
> to support Colorable, we will have only a few entries in the cache on a
> per project basis.  But...

I think the dependency data should be done on a per task basis and each task 
gets to decide how to handle it. We could set up a default directory to cache 
dependency information (think magic attributes like the classpath stuff for 
gump) ... not sure.

>
> > However we need to leave it up to the build writer how to store this
> > cache
>
> ...the user may specify it as well?

yep. It is only really necessary when the projects are medium to HUGE. It is 
something that Build Engineers should be able to twiddle for specific 
environments.

> > Unfortunately that only works if all the files were compiled
> > successfully. If not you have to figure out which ones were updated and
> > which failed and update dependency scanner/cache appropriately.
>
> This would become mute if we throw the files out of the picture.  But
> again, my fundamental idea of not considering files may be dead wrong. ;-) 
> So, please provide your views.

Well I guess we could separate it out into multiple layers/stages. I would 
still like the dependency cache file have "room" for the structural 
dependencies though.

> > However this level of sophistication may have to wait till ant2 because
> > it is a lot of work.
> >
> :-(


That is of course unless you want to do the work (and there is a damn lot of 
it - not an easy bit of coding IMHO), while still being backward compatible. 
If so ... woohooo ! ;)

-- 
Cheers,

Pete

*-----------------------------------------------------*
| Never argue with an idiot, they'll drag you down to |
| their level, and beat you with experience           |
*-----------------------------------------------------*

--
To unsubscribe, e-mail:   <mailto:ant-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:ant-dev-help@jakarta.apache.org>


Mime
View raw message