ant-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Antoine Levy-Lambert" <>
Subject Re: Using Ant to Collect Data?
Date Thu, 22 May 2003 21:06:25 GMT
It sounds like a reasonable idea.
ant seems to contain all or most of the tasks that you need, and you can
develop custom tasks as you said.
Check the execution speed though.
----- Original Message -----
From: "Robert Rice" <>
To: <>
Sent: Thursday, May 22, 2003 8:31 PM
Subject: Using Ant to Collect Data?

> I had a crazy thought today, that I wanted to toss out there for some
> I have stepped into a project that has a data collector component.  Data
> collected in a number of different forms, and a number of different ways.
> The process for gathering data is unique to a particular data feed.  These
> feeds change over time.  New feeds are added.  Old feeds are dropped.  The
> tasks involved in a data feed vary from feed to feed.  Here are a couple
> examples:
> 1: Get images from data provider A
>   a. download (ftp) a manifest that describes the available images
>   b. look at the manifest for images that match certain criteria
>   c. download (ftp) those selected images to an image server
> 2: Get images from data provider B
>   a.  execute a query that describes a set of available images
>   b. download (http) those selected images to an image server
> 3: TabDelim data feed from data provider C
>   a. download (ftp) a tab delimeted file from the provider
>   b. perform a basic validity check against the data ( correct number of
> columns )
>   c. move the file to the data process server ( where it will be parsed
> loaded to a database ).
> There are about 30 different feeds.  Scripts to perform the current feeds
> been constructed over time in a number of different languages.  There is
> little inhouse knowledge about these scripts.  The scripts are not
> maintainable/configurable.
> I've been tasked with adding a new data feed, but also keeping my eye on
> possibly bringing some of these other feeds under the same umbrella.
> As I installed Ant to manage the build of my codebase ( Java ) to
> this new feed, I wondered if Ant might be a good candidate to complete the
> original tasks?  In other words, try to use Ant as the data collector,
> coding custom tasks where totally necessary.
> I see a number of advantages in using Ant, but I may be totally out of
> 1. highly configurable through an XML file
> 2. plugin architecture for extending the functionality through Java
> 3. task based methodology, highly responsive to the ever changing business
> 4. lower level tasks can be tested independently of higher level tasks
> Is this a totally crazy idea?
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message