lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clemens Marschner" <c...@lanlab.de>
Subject Re: LARM Web Crawler: note on normalized URLs
Date Thu, 20 Jun 2002 14:16:37 GMT

> It may be even nicer to use some DB implemented in Java, such as
> HyperSQL (I think that's the name) or Smyle
> (https://sourceforge.net/projects/smyle/) or Berkeley DB
> (http://www.sleepycat.com/), although MySQL may be simpler if you want
> to create a crawler that can be run on a cluster of machines that share
> a central link repository.

Hm, I'll think about it. But MySQL seems to be the KISS way...
I don't think a central link repository makes sense. Looks like a bottleneck
to me.

Clemens


--
To unsubscribe, e-mail:   <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>


Mime
View raw message