ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dominique Devienne" <DDevie...@lgc.com>
Subject RE: cvs commit: ant/src/testcases/org/apache/tools/ant/taskdefs SyncTest.java
Date Thu, 20 Jan 2005 22:06:31 GMT
> From: Stefan Bodewig [mailto:bodewig@apache.org]
> 
> On 20 Jan 2005, <bodewig@apache.org> wrote:
> 
> >   Allow <sync> to keep files in target even if they are not in any
> >   source directories, PR 21832
> 
> Two things:

Let me address the second one first:

> (2) Names again.
> 
>     <sync todir="${dest}">
>       <fileset dir="${src}"/>
>       <deletefromtarget>
>         <exclude name="e/f"/>
>       </deletefromtarget>
>     </sync>
> 
> <deletefromtarget> is a <fileset> that doesn't allow dir to be
> set.  It implicitly contains <exclude>s for all the files that
> have been matched by one of the source-<fileset> (everything that
> should be there).  It can be used to keep things like logfiles or
> config files or similar in you target directory.  Default is to
> delete everything not present in any of the source sets.

Why a dir-less fileset rather than a patternset/selector?
What happens if I refid a fileset with a dir? It barfs?

> Now, is <deletefromtarget> a good name?  Is there a better one to
> capture what it does?

I don't know... I was kind of thinking the reverse, where you
specify files not to touch on the target side, like

	<preservedtargets>
        <include name="e/f" />
	</preservedtargets>

The advantage of this is you can do stuff like:

	<preservedtargets>
        <include name="**/*.log" />
        <exclude name="**/tmp/*.log" />
	</preservedtargets>

Which you cannot do with the <exclude> logic.
(although you can do it with selectors).

> (1) Dominique, could you please try a build of CVS HEAD (doesn't
> necessarily have to be any more recent than a month or so) on one
> of your larger sync targets to see whether <sync> has slowed down?
> 
> I have patched DirectoryScanner to optimize non-pattern matches in
> December, so the performance problems you expected should have
> been addressed.

I evaluated relative performance empirically: I used the latest
CVS HEAD and my own lsync, from which sync was originally based
on. I tried it on two filesets, a very small one of 5 sources
files, and in a big one (14K+ files, >140MB). In each case, I
measured the copy, and then a do nothing sync. I also tried at
the end to swap the order in which sync/lsync were used to
minimize caching interference.

So on the small fileset, the new sync takes 1.2s, v.s. 0.4s for
my old lsync.

For the big fileset, of course the initial copy takes a long
time (my system is getting old too), and sync was again slower
than lsync (in part because of caching probably), but for the
do nothing sync, it's still 44s for sync, and 32s for lsync.

A 33% slowdown with your optimizations doesn't sound too bad
for the added flexibility ;-) Cheers, --DD

PS: Below is the actual timing data.














P:\com_lgc\cycle9>P:\oss\org_apache\ant\apache-ant-1.7alpha\bin\ant -f
sync.xml -listener com.lgc.
buildmagic.util.TimerListener
Buildfile: sync.xml

sb-sync:
    [mkdir] Created dir: P:\com_lgc\cycle9\build\sb-sync
     [sync] Copying 5 files to P:\com_lgc\cycle9\build\sb-sync

dd-sync:
    [mkdir] Created dir: P:\com_lgc\cycle9\build\dd-sync
 [bm:lsync] Copying 5 files to P:\com_lgc\cycle9\build\dd-sync

all:

BUILD SUCCESSFUL
Total time: 2 seconds

* Timing by targets:
  1,204.0 ms  54.7% 1x sb-sync
    390.0 ms  17.7% 1x dd-sync
      0.0 ms   0.0% 1x all

* Timing by tasks:
  1,125.0 ms  51.1% 1x sync
    390.0 ms  17.7% 1x bm:lsync
    219.0 ms   9.9% 1x antlib
     79.0 ms   3.6% 2x mkdir

* Timing totals:
  2,203.0 ms 100.0% 1x total build time
  1,594.0 ms  72.4% 3x targets run
  1,813.0 ms  82.3% 5x tasks executed

  (% given of total build time)

P:\com_lgc\cycle9>P:\oss\org_apache\ant\apache-ant-1.7alpha\bin\ant -f
sync.xml -listener com.lgc.
buildmagic.util.TimerListener
Buildfile: sync.xml

sb-sync:

dd-sync:

all:

BUILD SUCCESSFUL
Total time: 2 seconds

* Timing by targets:
  1,125.0 ms  54.5% 1x sb-sync
    328.0 ms  15.9% 1x dd-sync
      0.0 ms   0.0% 1x all

* Timing by tasks:
  1,047.0 ms  50.8% 1x sync
    312.0 ms  15.1% 1x bm:lsync
    234.0 ms  11.3% 1x antlib
     78.0 ms   3.8% 2x mkdir

* Timing totals:
  2,063.0 ms 100.0% 1x total build time
  1,453.0 ms  70.4% 3x targets run
  1,671.0 ms  81.0% 5x tasks executed

  (% given of total build time)

P:\com_lgc\cycle9>REM Removed one source file from source
P:\com_lgc\cycle9>P:\oss\org_apache\ant\apache-ant-1.7alpha\bin\ant -f
sync.xml -listener com.lgc.
buildmagic.util.TimerListener
Buildfile: sync.xml

sb-sync:
     [sync] Removed 1 dangling file from P:\com_lgc\cycle9\build\sb-sync
     [sync] Removed 1 empty directory from
P:\com_lgc\cycle9\build\sb-sync

dd-sync:
 [bm:lsync] Removed 1 dangling file from P:\com_lgc\cycle9\build\dd-sync
 [bm:lsync] Removed 1 empty directory from
P:\com_lgc\cycle9\build\dd-sync

all:

BUILD SUCCESSFUL
Total time: 2 seconds

* Timing by targets:
  1,141.0 ms  55.8% 1x sb-sync
    328.0 ms  16.0% 1x dd-sync
      0.0 ms   0.0% 1x all

* Timing by tasks:
  1,063.0 ms  52.0% 1x sync
    328.0 ms  16.0% 1x bm:lsync
    219.0 ms  10.7% 1x antlib
     78.0 ms   3.8% 2x mkdir

* Timing totals:
  2,046.0 ms 100.0% 1x total build time
  1,469.0 ms  71.8% 3x targets run
  1,688.0 ms  82.5% 5x tasks executed

  (% given of total build time)

P:\com_lgc\cycle9>REM BIG fileset
P:\com_lgc\cycle9>P:\oss\org_apache\ant\apache-ant-1.7alpha\bin\ant -f
sync.xml -listener com.lgc.
buildmagic.util.TimerListener -Dfrom=C:\pro\jdk1.4-src
Buildfile: sync.xml

sb-sync:
    [mkdir] Created dir: P:\com_lgc\cycle9\build\sb-sync
     [sync] Copying 14231 files to P:\com_lgc\cycle9\build\sb-sync

dd-sync:
    [mkdir] Created dir: P:\com_lgc\cycle9\build\dd-sync
 [bm:lsync] Copying 14231 files to P:\com_lgc\cycle9\build\dd-sync

all:

BUILD SUCCESSFUL
Total time: 14 minutes 27 seconds

* Timing by targets:
  476,687.0 ms  55.0% 1x sb-sync
  389,875.0 ms  45.0% 1x dd-sync
       94.0 ms   0.0% 1x <top-level pseudo-target>
        0.0 ms   0.0% 1x all

* Timing by tasks:
  476,687.0 ms  55.0% 1x sync
  389,875.0 ms  45.0% 1x bm:lsync
      235.0 ms   0.0% 1x antlib
       94.0 ms   0.0% 1x property
        0.0 ms   0.0% 2x mkdir

* Timing totals:
  867,297.0 ms 100.0% 1x total build time
  866,656.0 ms  99.9% 4x targets run
  866,891.0 ms 100.0% 6x tasks executed

  (% given of total build time)

P:\com_lgc\cycle9>
P:\com_lgc\cycle9>P:\oss\org_apache\ant\apache-ant-1.7alpha\bin\ant -f
sync.xml -listener com.lgc.
buildmagic.util.TimerListener -Dfrom=C:\pro\jdk1.4-src
Buildfile: sync.xml

sb-sync:

dd-sync:

all:

BUILD SUCCESSFUL
Total time: 1 minute 16 seconds

* Timing by targets:
  45,594.0 ms  59.7% 1x sb-sync
  30,078.0 ms  39.4% 1x dd-sync
      94.0 ms   0.1% 1x <top-level pseudo-target>
       0.0 ms   0.0% 1x all

* Timing by tasks:
  45,578.0 ms  59.7% 1x sync
  30,078.0 ms  39.4% 1x bm:lsync
     203.0 ms   0.3% 1x antlib
      94.0 ms   0.1% 1x property
      16.0 ms   0.0% 2x mkdir

* Timing totals:
  76,375.0 ms 100.0% 1x total build time
  75,766.0 ms  99.2% 4x targets run
  75,969.0 ms  99.5% 6x tasks executed

  (% given of total build time)

P:\com_lgc\cycle9>
P:\com_lgc\cycle9>
P:\com_lgc\cycle9>
P:\com_lgc\cycle9>P:\oss\org_apache\ant\apache-ant-1.7alpha\bin\ant -f
sync.xml -listener com.lgc.
buildmagic.util.TimerListener -Dfrom=C:\pro\jdk1.4-src
Buildfile: sync.xml

dd-sync:

sb-sync:

all:

BUILD SUCCESSFUL
Total time: 1 minute 16 seconds

* Timing by targets:
  43,829.0 ms  57.3% 1x sb-sync
  31,984.0 ms  41.8% 1x dd-sync
      93.0 ms   0.1% 1x <top-level pseudo-target>
       0.0 ms   0.0% 1x all

* Timing by tasks:
  43,829.0 ms  57.3% 1x sync
  31,968.0 ms  41.8% 1x bm:lsync
     360.0 ms   0.5% 1x antlib
      93.0 ms   0.1% 1x property
      16.0 ms   0.0% 2x mkdir

* Timing totals:
  76,516.0 ms 100.0% 1x total build time
  75,906.0 ms  99.2% 4x targets run
  76,266.0 ms  99.7% 6x tasks executed

  (% given of total build time)

P:\com_lgc\cycle9>REM Reversed sync/lsync to lsync/sync
P:\com_lgc\cycle9>P:\oss\org_apache\ant\apache-ant-1.7alpha\bin\ant -f
sync.xml -listener com.lgc.
buildmagic.util.TimerListener -Dfrom=C:\pro\jdk1.4-src
Buildfile: sync.xml

dd-sync:

sb-sync:

all:

BUILD SUCCESSFUL
Total time: 1 minute 16 seconds

* Timing by targets:
  43,687.0 ms  57.3% 1x sb-sync
  31,922.0 ms  41.8% 1x dd-sync
      94.0 ms   0.1% 1x <top-level pseudo-target>
       0.0 ms   0.0% 1x all

* Timing by tasks:
  43,687.0 ms  57.3% 1x sync
  31,906.0 ms  41.8% 1x bm:lsync
     344.0 ms   0.5% 1x antlib
      94.0 ms   0.1% 1x property
      16.0 ms   0.0% 2x mkdir

* Timing totals:
  76,297.0 ms 100.0% 1x total build time
  75,703.0 ms  99.2% 4x targets run
  76,047.0 ms  99.7% 6x tasks executed

  (% given of total build time)

P:\com_lgc\cycle9>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message