lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clemens Marschner" <>
Subject Re: LARM Web Crawler: note on normalized URLs
Date Thu, 20 Jun 2002 14:16:37 GMT

> It may be even nicer to use some DB implemented in Java, such as
> HyperSQL (I think that's the name) or Smyle
> ( or Berkeley DB
> (, although MySQL may be simpler if you want
> to create a crawler that can be run on a cluster of machines that share
> a central link repository.

Hm, I'll think about it. But MySQL seems to be the KISS way...
I don't think a central link repository makes sense. Looks like a bottleneck
to me.


To unsubscribe, e-mail:   <>
For additional commands, e-mail: <>

View raw message