cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vadim Gritsenko" <vadim.gritse...@verizon.net>
Subject RE: Crawler/Indexer redesign
Date Mon, 04 Feb 2002 22:37:05 GMT
> From: Bernhard Huber [mailto:berni_huber@a1.net]
> 
> hi,
> 
> >How about
> >
> >  Collection crawl(Source)
> >
> >? Then crawler can be ThreadSafe.
> >
> Yes, it would be ThreadSafe, storing all crawled resources in the
> collection.
> Does this work for crawling huge sites?
> 
> My idea was to handle that problem by introducing the Iterator.
> Using Iterator might allow to process some crawled resources quite
early.
> Using collection might delay the processing of the crawled resources
> until the crawling has terminated,
> that might take quite some time.
> 
> Hence it might be better:
> Iterator crawl( Source)

Go for it. Just make sure you are not buffering results from this
Iterator somewhere down the pipe ;)

Vadim

> 
> bye bernhard


---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org


Mime
View raw message