ant-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Antoine Levy-Lambert" <levylamb...@tiscali-dsl.de>
Subject Re: Using Ant to Collect Data?
Date Thu, 22 May 2003 21:06:25 GMT
It sounds like a reasonable idea.
ant seems to contain all or most of the tasks that you need, and you can
develop custom tasks as you said.
Check the execution speed though.
Antoine
----- Original Message -----
From: "Robert Rice" <robert@windermere.com>
To: <user@ant.apache.org>
Sent: Thursday, May 22, 2003 8:31 PM
Subject: Using Ant to Collect Data?


> I had a crazy thought today, that I wanted to toss out there for some
feedback.
>
> I have stepped into a project that has a data collector component.  Data
is
> collected in a number of different forms, and a number of different ways.
>
> The process for gathering data is unique to a particular data feed.  These
> feeds change over time.  New feeds are added.  Old feeds are dropped.  The
> tasks involved in a data feed vary from feed to feed.  Here are a couple
of
> examples:
>
> 1: Get images from data provider A
>   a. download (ftp) a manifest that describes the available images
>   b. look at the manifest for images that match certain criteria
>   c. download (ftp) those selected images to an image server
>
> 2: Get images from data provider B
>   a.  execute a query that describes a set of available images
>   b. download (http) those selected images to an image server
>
> 3: TabDelim data feed from data provider C
>   a. download (ftp) a tab delimeted file from the provider
>   b. perform a basic validity check against the data ( correct number of
> columns )
>   c. move the file to the data process server ( where it will be parsed
and
> loaded to a database ).
>
> There are about 30 different feeds.  Scripts to perform the current feeds
have
> been constructed over time in a number of different languages.  There is
> little inhouse knowledge about these scripts.  The scripts are not
> maintainable/configurable.
>
> I've been tasked with adding a new data feed, but also keeping my eye on
> possibly bringing some of these other feeds under the same umbrella.
>
> As I installed Ant to manage the build of my codebase ( Java ) to
construct
> this new feed, I wondered if Ant might be a good candidate to complete the
> original tasks?  In other words, try to use Ant as the data collector,
only
> coding custom tasks where totally necessary.
>
> I see a number of advantages in using Ant, but I may be totally out of
line:
> 1. highly configurable through an XML file
> 2. plugin architecture for extending the functionality through Java
> 3. task based methodology, highly responsive to the ever changing business
needs
> 4. lower level tasks can be tested independently of higher level tasks
>
> Is this a totally crazy idea?
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
> For additional commands, e-mail: user-help@ant.apache.org
>


Mime
View raw message