Return-Path: Mailing-List: contact user-help@ant.apache.org; run by ezmlm Delivered-To: mailing list user@ant.apache.org Received: (qmail 3591 invoked from network); 2 Jun 2003 22:15:12 -0000 Received: from smtp-send.load.com (209.58.232.28) by daedalus.apache.org with SMTP; 2 Jun 2003 22:15:12 -0000 Received: (qmail 18271 invoked from network); 2 Jun 2003 22:15:18 -0000 Received: from unknown (HELO ?172.29.128.122?) (63.145.195.2) by smtp03.load.com with SMTP; 2 Jun 2003 22:15:18 -0000 Date: Mon, 2 Jun 2003 15:16:31 -0700 (PDT) From: Robert Rice X-X-Sender: robert@localhost.localdomain To: user@ant.apache.org Subject: Help getting a large number of files from an ftp server. Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N I've been working on an ant target that is required to get a number of files from an ftp server. So far, I've been using the task to do the work. I have it operating on an includesfile, that describes the files to download. This process seems to work rather well with a reasonably short list of files. A list of 10 files takes approx. 5 minutes. A list of 35 files takes approx 10 minutes. A list of 100 files takes approx 30 minutes. The problem is that my includesfile may have a couple thousand files to download. If I send this many files to the task, the server times out after about 4 hours, prior to downloading any files. Here is the error. BUILD FAILED file:/home/robert/dvl/collector/NorthwestImageCollector.xml:76: Error while communicating with FTP server: Total time: 243 minutes 21 seconds In all cases, success or otherwise, the task seems to hang for the majority of the processing time, prior to transferring any images. The logged message is shown below. ftp-images: [ftp] getting files If the process makes it past this sticky point, then it rather quickly downloads the requested files with the following example logged messages: [ftp] transferring webphoto/bigphoto/96/23010396_04.jpg to /home/robert/dvl/collector/build/images/Northwest/webphoto/bigphoto/96/23010396_04.jpg [ftp] transferring webphoto/bigphoto/96/23010396_05.jpg to /home/robert/dvl/collector/build/images/Northwest/webphoto/bigphoto/96/23010396_05.jpg Does anyone have suggestions on better ways to get a large number of files from an ftp server? Thanks, Robert Rice