Return-Path: 
 <dev-return-47466-apmail-ant-dev-archive=ant.apache.org@ant.apache.org>
Delivered-To: apmail-ant-dev-archive@ant.apache.org
Received: (qmail 83749 invoked by uid 500); 27 Jun 2003 16:05:16 -0000
Mailing-List: contact dev-help@ant.apache.org; run by ezmlm
Precedence: bulk
List-Unsubscribe: <mailto:dev-unsubscribe@ant.apache.org>
List-Subscribe: <mailto:dev-subscribe@ant.apache.org>
List-Help: <mailto:dev-help@ant.apache.org>
List-Post: <mailto:dev@ant.apache.org>
List-Id: "Ant Developers List" <dev.ant.apache.org>
Reply-To: "Ant Developers List" <dev@ant.apache.org>
Delivered-To: mailing list dev@ant.apache.org
Received: (qmail 83735 invoked by uid 500); 27 Jun 2003 16:05:16 -0000
Received: (qmail 83731 invoked from network); 27 Jun 2003 16:05:16 -0000
Received: from junior.lgc.com (134.132.72.99)
  by daedalus.apache.org with SMTP; 27 Jun 2003 16:05:16 -0000
Received: from lgchvw01.lgc.com (lgchvw01.lgc.com [134.132.93.107])
	by junior.lgc.com (8.11.7/8.11.3) with SMTP id h5RG49s23912
	for <ant-dev@jakarta.apache.org>; Fri, 27 Jun 2003 11:04:10 -0500 (CDT)
Received: from 134.132.93.152 by lgchvw01.lgc.com (InterScan E-Mail VirusWall
 NT); Fri, 27 Jun 2003 11:05:17 -0500
Received: by lgchexchbh.ad.lgc.com with Internet Mail Service (5.5.2653.19)
	id <M3YP7Y5S>; Fri, 27 Jun 2003 11:05:16 -0500
Message-ID: <D44A54C298394F4E967EC8538B1E00F10248C375@lgchexch002.lgc.com>
From: Dominique Devienne <DDevienne@lgc.com>
To: Ant Developers List <ant-dev@jakarta.apache.org>
Subject: About <get>, <copy>, and buffer sizes
Date: Fri, 27 Jun 2003 11:05:15 -0500
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain
X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N

Today I evaluated the various ways I could download a file efficiently. So I
played with <get>, using both the HTTP and FILE protocol, and with <copy>
(see build file below). The file I'm downloading is 38,448,695 bytes long.
The output of running each target twice is also shown below.

So what is this all about?

Well, the HTTP protocol is slower than the FILE protocol, but I expected
that, because the HTTP server is serving files that reside on another
NetApps filer machine (used to be faster when serving from its own disks).
Note though that even though the file: protocol is faster, is not "smart" as
implemented by <get>, i.e. it always redownloads, even when the local copy
is up-to-date. (note that the bigshot HTTP server is serving the same file
from afiler1).

I then tried out copy, which I know to be smart, and expected to be as fast
as <get> with the file: protocol. But to my surprise, it's not at all, and
is in fact more than TWICE slower!!!

A quick look at the code reveals that <get> uses a 100*1024 bytes buffer,
when <copy> uses a 8*1024 bytes buffer. This seems to have quite an impact
on download performance!!!

Given this information, I'd like:
1) <get> to be made "smarter" for the file protocol
2) <copy> to be made faster by adding the ability to control the buffer size
used, as was recently added to <checksum> I believe.

In my particular case, I'd prefer (1), to be able to switch easily from
http: to file: without having to change my build files (just the URL prefix
used). I'll probably try it myself soon. (2) would be good too, and is
probably simple to implement.

Just a head's up. I don't recall anybody noticing something like this.

Cheers, --DD

P:\com_lgc\buildtools>build -f get.xml get.http
Buildfile: get.xml

get.http:     [GET]
Getting:
http://bigshot.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip

BUILD SUCCESSFUL
Total time: 1 minute 7 seconds
P:\com_lgc\buildtools>build -f get.xml get.http
Buildfile: get.xml

get.http:     [GET]
Getting:
http://bigshot.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip
Not modified - so not downloaded

BUILD SUCCESSFUL
Total time: 1 second
P:\com_lgc\buildtools>build -f get.xml get.file
Buildfile: get.xml

get.file:     [GET]
Getting:
file://afiler1.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip

BUILD SUCCESSFUL
Total time: 25 seconds
P:\com_lgc\buildtools>build -f get.xml get.file
Buildfile: get.xml

get.file:     [GET]
Getting:
file://afiler1.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip

BUILD SUCCESSFUL
Total time: 25 seconds
P:\com_lgc\buildtools>build -f get.xml copy.file
Buildfile: get.xml

copy.file:    [GET]
Copying 1 file to P:\com_lgc\buildtools

BUILD SUCCESSFUL
Total time: 1 minute 8 seconds
P:\com_lgc\buildtools>build -f get.xml copy.file
Buildfile: get.xml

BUILD SUCCESSFUL
Total time: 1 second

<?xml version="1.0"?>

<project name="GET" default="help">

  <target name="help" description="Display project description and targets">
    <echo>=== ${ant.project.name} Build ===</echo>
    <java classname="org.apache.tools.ant.Main">
      <arg value="-emacs" />
      <arg value="-projecthelp" />
      <arg value="-buildfile" />
      <arg value="${ant.file}" />
    </java>
  </target>

  <property name="dsinfra/" value="dsinfra/HEAD/latest/module-dsinfra.zip"
/>
  <property name="dsinfra\" value="dsinfra\HEAD\latest\module-dsinfra.zip"
/>

  <target name="get.http" description="Get dsinfra.zip, HTTP protocol">
    <get src="http://bigshot.ad.lgc.com/dsbuilds/${dsinfra/}"
         dest="dsinfra-http.zip"
         useTimeStamp="true" />
  </target>

  <target name="get.file" description="Get dsinfra.zip, FILE protocol">
    <get src="file:////afiler1.ad.lgc.com/dsbuilds/${dsinfra/}"
         dest="dsinfra-file.zip"
         useTimeStamp="true" />
  </target>

  <target name="copy.file" description="Copy dsinfra.zip">
    <copy file="\\afiler1.ad.lgc.com\dsbuilds\${dsinfra\}"
          tofile="dsinfra-copy.zip"
          preserveLastModified="true" />
  </target>

</project>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org