ant-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject [Bug 62849] DirectoryScanner does not efficiently handle filesystem loops
Date Sun, 09 Dec 2018 19:51:14 GMT
https://bz.apache.org/bugzilla/show_bug.cgi?id=62849

--- Comment #12 from Michael Barker <mikeb01@gmail.com> ---
The current scenario we are running into:

- We have a build with roughly the following structure.  This is an artefact of
the build tool we happen to be using (BUCK).

<project root>
  <source directory>
  ...
  <source directory>
  <build output>
     <symlink -> project root> A
     <output directory>
        <symlink -> A>
     ...
     <output directory>
        <symlink -> A>

The build has a total of around 1M files, of which ~800K are in the output
directory.  Note that we don't use Ant to build any of this.  Where this hits
us is that we use Jenkins for CI  and the Ant DirectoryScanner is used find
artefacts from the build to archive.

When Jenkins runs the scanner is continues to recurse the project through the
project root symlink.  Because the DirectoryScanner holds onto all of the files
it visits (not just the ones that match) it rapidly builds a very large list of
files.  Some heap debugging shows ~50M entries in the directory scanner before
the Jenkins archive task starts to fail with OOM errors.

I'll have a look later this week about trying to build a test case for this.

-- 
You are receiving this mail because:
You are the assignee for the bug.
Mime
View raw message