ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominique Devienne <DDevie...@lgc.com>
Subject About <get>, <copy>, and buffer sizes
Date Fri, 27 Jun 2003 16:05:15 GMT
Today I evaluated the various ways I could download a file efficiently. So I
played with <get>, using both the HTTP and FILE protocol, and with <copy>
(see build file below). The file I'm downloading is 38,448,695 bytes long.
The output of running each target twice is also shown below.

So what is this all about?

Well, the HTTP protocol is slower than the FILE protocol, but I expected
that, because the HTTP server is serving files that reside on another
NetApps filer machine (used to be faster when serving from its own disks).
Note though that even though the file: protocol is faster, is not "smart" as
implemented by <get>, i.e. it always redownloads, even when the local copy
is up-to-date. (note that the bigshot HTTP server is serving the same file
from afiler1).

I then tried out copy, which I know to be smart, and expected to be as fast
as <get> with the file: protocol. But to my surprise, it's not at all, and
is in fact more than TWICE slower!!!

A quick look at the code reveals that <get> uses a 100*1024 bytes buffer,
when <copy> uses a 8*1024 bytes buffer. This seems to have quite an impact
on download performance!!!

Given this information, I'd like:
1) <get> to be made "smarter" for the file protocol
2) <copy> to be made faster by adding the ability to control the buffer size
used, as was recently added to <checksum> I believe.

In my particular case, I'd prefer (1), to be able to switch easily from
http: to file: without having to change my build files (just the URL prefix
used). I'll probably try it myself soon. (2) would be good too, and is
probably simple to implement.

Just a head's up. I don't recall anybody noticing something like this.

Cheers, --DD

P:\com_lgc\buildtools>build -f get.xml get.http
Buildfile: get.xml

get.http:     [GET]
Getting:
http://bigshot.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip

BUILD SUCCESSFUL
Total time: 1 minute 7 seconds
P:\com_lgc\buildtools>build -f get.xml get.http
Buildfile: get.xml

get.http:     [GET]
Getting:
http://bigshot.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip
Not modified - so not downloaded

BUILD SUCCESSFUL
Total time: 1 second
P:\com_lgc\buildtools>build -f get.xml get.file
Buildfile: get.xml

get.file:     [GET]
Getting:
file://afiler1.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip

BUILD SUCCESSFUL
Total time: 25 seconds
P:\com_lgc\buildtools>build -f get.xml get.file
Buildfile: get.xml

get.file:     [GET]
Getting:
file://afiler1.ad.lgc.com/dsbuilds/dsinfra/HEAD/latest/module-dsinfra.zip

BUILD SUCCESSFUL
Total time: 25 seconds
P:\com_lgc\buildtools>build -f get.xml copy.file
Buildfile: get.xml

copy.file:    [GET]
Copying 1 file to P:\com_lgc\buildtools

BUILD SUCCESSFUL
Total time: 1 minute 8 seconds
P:\com_lgc\buildtools>build -f get.xml copy.file
Buildfile: get.xml

BUILD SUCCESSFUL
Total time: 1 second

<?xml version="1.0"?>

<project name="GET" default="help">

  <target name="help" description="Display project description and targets">
    <echo>=== ${ant.project.name} Build ===</echo>
    <java classname="org.apache.tools.ant.Main">
      <arg value="-emacs" />
      <arg value="-projecthelp" />
      <arg value="-buildfile" />
      <arg value="${ant.file}" />
    </java>
  </target>

  <property name="dsinfra/" value="dsinfra/HEAD/latest/module-dsinfra.zip"
/>
  <property name="dsinfra\" value="dsinfra\HEAD\latest\module-dsinfra.zip"
/>

  <target name="get.http" description="Get dsinfra.zip, HTTP protocol">
    <get src="http://bigshot.ad.lgc.com/dsbuilds/${dsinfra/}"
         dest="dsinfra-http.zip"
         useTimeStamp="true" />
  </target>

  <target name="get.file" description="Get dsinfra.zip, FILE protocol">
    <get src="file:////afiler1.ad.lgc.com/dsbuilds/${dsinfra/}"
         dest="dsinfra-file.zip"
         useTimeStamp="true" />
  </target>

  <target name="copy.file" description="Copy dsinfra.zip">
    <copy file="\\afiler1.ad.lgc.com\dsbuilds\${dsinfra\}"
          tofile="dsinfra-copy.zip"
          preserveLastModified="true" />
  </target>

</project>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message