Return-Path: 
 <dev-return-83279-apmail-ant-dev-archive=ant.apache.org@ant.apache.org>
Delivered-To: apmail-ant-dev-archive@www.apache.org
Received: (qmail 53562 invoked from network); 19 Sep 2008 07:32:00 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 19 Sep 2008 07:32:00 -0000
Received: (qmail 47487 invoked by uid 500); 19 Sep 2008 07:31:56 -0000
Delivered-To: apmail-ant-dev-archive@ant.apache.org
Received: (qmail 47376 invoked by uid 500); 19 Sep 2008 07:31:56 -0000
Mailing-List: contact dev-help@ant.apache.org; run by ezmlm
Precedence: bulk
List-Unsubscribe: <mailto:dev-unsubscribe@ant.apache.org>
List-Help: <mailto:dev-help@ant.apache.org>
List-Post: <mailto:dev@ant.apache.org>
List-Id: "Ant Developers List" <dev.ant.apache.org>
Reply-To: "Ant Developers List" <dev@ant.apache.org>
Delivered-To: mailing list dev@ant.apache.org
Received: (qmail 47365 invoked by uid 99); 19 Sep 2008 07:31:56 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 19 Sep 2008 00:31:56 -0700
X-ASF-Spam-Status: No, hits=1.2 required=10.0
	tests=SPF_NEUTRAL
X-Spam-Check-By: apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Received: from [88.84.142.181] (HELO samaflost.de) (88.84.142.181)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 19 Sep 2008 07:30:58 +0000
Received: from localhost (unknown [127.0.0.1])
	by samaflost.de (Postfix) with ESMTP id 831FDE220014
	for <dev@ant.apache.org>; Fri, 19 Sep 2008 07:31:30 +0000 (UTC)
Received: from samaflost.de ([127.0.0.1])
 by localhost (v30161.1blu.de [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id 32480-10 for <dev@ant.apache.org>;
 Fri, 19 Sep 2008 09:31:28 +0200 (CEST)
Received: by samaflost.de (Postfix, from userid 2525)
	id DDB8BE22000A; Fri, 19 Sep 2008 09:31:28 +0200 (CEST)
To: dev@ant.apache.org
Subject: Re: DirectoryScanner performance
X-Draft-From: ("nnfolder:mail.jakarta-ant" 89331)
References: <y1uskrxixkc.fsf@v30161.1blu.de>
	<ab8e556c0809180309q733a872ao638120550218c171@mail.gmail.com>
From: Stefan Bodewig <bodewig@apache.org>
Date: Fri, 19 Sep 2008 09:31:28 +0200
In-Reply-To: <ab8e556c0809180309q733a872ao638120550218c171@mail.gmail.com>
 (Kevin Jackson's message of "Thu\, 18 Sep 2008 11\:09\:53 +0100")
Message-ID: <y1utzcclhgv.fsf@v30161.1blu.de>
User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Virus-Scanned: amavisd-new at samaflost.de
X-Virus-Checked: Checked by ClamAV on apache.org

On Thu, 18 Sep 2008, Kevin Jackson <foamdino@gmail.com> wrote:

>> The tests are not through yet, but one thing was so surprising to
>> me that I wanted to show it upfront:
>>
>> Running the matchall target (of sr/etc/performance/dirscanner.xml):
>>
>> Ant 1.6.5               1 min 30 s      ~ 19 MB
>> Ant 1.7.0               3 min 53 s      ~ 24 MB
>> Ant 1.7.1                     10 s      ~ 14 MB
>>
>> So 1.7.0 took more than twice as long as 1.6.5 to find all files in
>> a big directory tree without any patterns and 1.7.1 is a whole lot
>> faster than even 1.6.5.
> 
> That's a huge difference - what are we doing now in 1.7.1 that is
> different from before?

Apart from the I/O syscall reducing change there is another major
change in Union (svn rev 581394) which is similar in effect to my
VectorSet change in trunk's HEAD - avoid the linear time complexity of
List.contains().

If I change the scan macrodef in dirscanner.xml to read

  <macrodef name="scan">
    <attribute name="test"/>
    <element name="patterns" optional="true"/>
    <sequential>
      <pathconvert property="@{test}">
          <fileset dir="${test.dir}" followSymlinks="${symlinks}"
                   casesensitive="${casesensitive}">
            <patterns/>
          </fileset>
      </pathconvert>
    </sequential>
  </macrodef>

i.e. remove the <path> around <fileset> that was only there to allow
it to work in 1.6.5, then I avoid Union's code and the results become

Ant 1.7.0                     45 s      ~ 18 MB
Ant 1.7.1                      9 s      ~ 18 MB
svn rev 696674                 4 s      ~ 41 MB

for the matchall target (same setup as for the quoted results).

So directory scanning performance itself hasn't degraded as much from
1.6.5 to 1.7.0 (it may even have become faster) but evaluating a
<fileset> inside a <path> has.

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org