lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <>
Subject Re: Configuration RFC
Date Sun, 14 Jul 2002 03:06:12 GMT

> This comes up when there is a MVC url methodology or a URL with POST
> parameters.
> So /app1/ShowResults
> Could show lots of different results depending on what were the
> parameters passed.

This wouldn't really apply to a crawler such as LARM, as it discovers
and follows only URLs it finds in fetched documents.  It ignores HTML
forms, vairous form fields, etc., does not POST, just GETs links that
it finds and that pass the filtering criteria.

This should also answer the other question about directly fetching
links that match a pattern.  Most crawlers are designed to fetch pages,
extract HTML, links, images, etc., keep some, throw away some, fetch
some more, and so on.


Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes

To unsubscribe, e-mail:   <>
For additional commands, e-mail: <>

View raw message