lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Carlson <carl...@bookandhammer.com>
Subject Re: Configuration RFC
Date Sun, 14 Jul 2002 04:38:58 GMT
Otis,

Many site are using this MVC methodology to create their sites. Why develop
a new crawler with the same limitations?

--Peter


On 7/13/02 8:06 PM, "Otis Gospodnetic" <otis_gospodnetic@yahoo.com> wrote:

> Peter,
> 
>> This comes up when there is a MVC url methodology or a URL with POST
>> parameters.
>> So /app1/ShowResults
>> 
>> Could show lots of different results depending on what were the
>> parameters passed.
> 
> This wouldn't really apply to a crawler such as LARM, as it discovers
> and follows only URLs it finds in fetched documents.  It ignores HTML
> forms, vairous form fields, etc., does not POST, just GETs links that
> it finds and that pass the filtering criteria.
> 
> This should also answer the other question about directly fetching
> links that match a pattern.  Most crawlers are designed to fetch pages,
> extract HTML, links, images, etc., keep some, throw away some, fetch
> some more, and so on.
> 
> Otis
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Autos - Get free new car price quotes
> http://autos.yahoo.com
> 
> --
> To unsubscribe, e-mail:   <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>
> 
> 


--
To unsubscribe, e-mail:   <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>


Mime
View raw message