ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Bodewig <bode...@apache.org>
Subject Re: DirectoryScanner Memory Consumption
Date Mon, 22 Sep 2008 08:07:42 GMT
On Fri, 19 Sep 2008, Stefan Bodewig <bodewig@apache.org> wrote:

> Something in File.getCanonicalPath seems to consume memory that
> isn't easily reclaimed by the GC, and I have no idea what we could
> do about it.

except increasing the number of times we are allowed to traverse
directories of the same name before we check whether they are part of
a symlink cycle.  svn revision 697692 bumps it to five instead of one.

Just to show where we've come now (and I think I'll stop at this
point):

small setup
===========

matchall using pathconvert
--------------------------

Ant 1.6.5        1 min 30 s  ~ 19 MB
Ant 1.7.0        3 min 53 s  ~ 24 MB
Ant 1.7.1               9 s  ~ 21 MB
svn rev 697073          3 s  ~ 26 MB
svn rev 697692          2 s  ~ 19 MB

matchall using resourcecount
----------------------------

Ant 1.7.0        1 min 25 s  ~ 21 MB
Ant 1.7.1               8 s  ~ 14 MB
svn rev 697073          2 s  ~ 17 MB
svn rev 697692          2 s  ~ 16 MB

big setup
=========

matchall using resourcecount
----------------------------

Ant 1.7.1       14 min  2 s  ~ 55 MB
svn rev 697073         53 s  ~ 78 MB
svn rev 697692         22 s  ~ 64 MB

"small" is 10000 directories and about 22000 files spread accross four
levels, "big" is one level more and an order of magnitude of files and
dirs more.

The numbers differ a bit from the last I've posted because I had
ANT_OPTS=-Xmx512m set last time, this time svn rev 607073 consumes
less memory but takes longer while I'm reasonably happy with speed and
memory consumption of svn rev 697692.

Overall we've roughly gone from quadratic time to linear time when
scanning directories - and we even scan small directory trees faster
than 1.7.1.

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message