forrest-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicola Ken Barozzi <>
Subject Re: [RT] crawl our dynamic forrest rather than commandline
Date Fri, 02 Sep 2005 07:15:50 GMT
David Crossley wrote:
> Perhaps i am not explaining my concept very well.
> Anyway, i have a new one. Forget wget and use
> our own Cocoon capabilities.
> I wonder if we can make a special pipeline in Forrest
> that does the following:
> * crawls the dynamic server (i.e. crawls itself)
> * determines each file type (by using the mime-type
> that Forrest indicates and perhaps also a map of hints)
> * transforms each document to rewrite the
> links (e.g. howto/foobar => howto/foobar.html)
> * use the Cocoon SourceWritingTransformer to
> write each file to disk with the relevant filename
> extension.

You are describing the CLI, only that the CLI uses an extra Cocoon
instance. If you look inside the implementation you will se all the
above, albeit with some bugs/unfinished parts. It just needs fixing.

Nicola Ken Barozzi         
            - verba volant, scripta manent -
   (discussions get forgotten, just code remains)

View raw message