ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Rice <rob...@windermere.com>
Subject Re: AW: Issues with <modified /> selector.
Date Wed, 23 Jun 2004 19:53:44 GMT
 have created a patch that addresses the poor cachefile save performance.  
This patch addresses this bug, as well as bug 29742.

The revised ModifiedSelector now implements the BuildListener interface.  
Upon configuration, it registers itself with the Project as a 
BuildListener.  The ModifiedSelector is notified of finished BuildEvents, 
like taskFinished, targetFinished and buildFinished.  Saving of the 
cachefile is now delayed until the next finished BuildEvent occurs, most 
likely, a taskFinished call.

There is also a new attribute called "delayupdate".  It defaults to true 
to gain the new performance increase of delaying the save of the 
cachefile.  Setting the "delayupdate" attribute to false allows continual 
cachefile updates ( like the current ModifiedSelector ).

I also ran a sample population of files to note the reduction in time to 
cache the file changes:

sample: 6596 image files averaging 31K, total size of 204 MB
       the cache is empty, so all files are seen as revised,
       this is the most extreme case to show the largest performance gain

MD5 digest mode with continual update:  9 minutes, 8 seconds
MD5 digest mode with delayed update:    36 seconds
CRC checksum mode with continual update: 7 minutes, 36 seconds
CRC checksum mode with delayed update: 43 seconds

On Wed, 23 Jun 2004 Jan.Materne@rzf.fin-nrw.de wrote:

> I´ll have a deeper look into that
> (but now I have to ensure that my new computer is able to run - the old was
> damaged :(
> 
> 
> Jan
> 
> 
> > -----Ursprüngliche Nachricht-----
> > Von: Robert Rice [mailto:robert@windermere.com]
> > Gesendet am: Dienstag, 22. Juni 2004 20:23
> > An: dev@ant.apache.org
> > Betreff: Issues with <modified /> selector.
> > 
> > Recently, I submitted two bugs related to the modified selector.
> > 
> > Bug 29742 addresses the issue that the Modified selector 
> > doesn't allow 
> > algorithm or comparator selection.  Only the default choices 
> > of "digest" 
> > and "equal" respectively are allowed.  
> > 
> > I have created a bug fix for the modified selector that 
> > addresses this 
> > issue. 
> > In the bug fix, the algorithm and comparator initialization 
> > logic has been
> > changed to allow choosing away from default.
> > 
> > At the same time, I have created another algorithm type, 
> > "checksum", that 
> > makes
> > use of the java.util.zip.Checksum interface.  It is chosen by 
> > setting the
> > alogrithm attribute to "checksum" ( <modified 
> > algorithm="checksum" /> ).  
> > 
> > This checksum allows the choice of CRC32 or Alder32, which utilize
> > java.util.zip.CRC32 and java.util.zip.Adler32 respectively.  
> > This choice 
> > is made
> > by setting the algorithm.algorithm parameter to "CRC" or 
> > "ADLER", with the
> > default being "CRC" ( <modified algorithm="checksum"><param
> > name="algorithm.algorithm" value="ADLER" /></modified> )
> > 
> > How do I submit such changes for evaluation and possible inclusion?
> > 
> > Bug 29743 addresses a bigger issue in that the modified 
> > selector has poor 
> > cachefile save performance.  The architecture of the selector 
> > framework 
> > dictates that the selector is notified of files, one at a 
> > time, through 
> > the isSelected method.  It is at this method call, that the cache 
> > properties file is saved.  This properties file is saved every time a 
> > file modification is found, possibly as many times as there 
> > are files in 
> > the fileset.  This leads to poor performance when process 
> > large filesets 
> > with multiple changes, in my case around 15000.
> > 
> > I would like to delay the saving of the file until we are 
> > finished with 
> > the fileset or finished with the selector.  It looks to me, like this 
> > requires the addition of one of more methods to the selector 
> > framework, 
> > likely to BaseSelector. 
> > 
> > In the case of the modified selector, we may be able to get away with 
> > knowing we are finished with the selector through some method 
> > call like 
> > "tearDown".  At this point, the modified selector could save its 
> > properites file.
> > 
> > Another option is to add filesetListener type of behavior to 
> > the BaseSelector.
> > This would notify a selector when a fileset selection has started and 
> > ended. With this option, the modified selector could save its 
> > properties 
> > file at the end of fileset selection.
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> > For additional commands, e-mail: dev-help@ant.apache.org
> > 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message