lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <>
Subject Re: Is Lucene right for me ?
Date Thu, 26 Aug 2004 11:33:38 GMT

Yes, you can use Lucene for this.  10 mil. documents is not much, if
you use adequate hardware.  You can use boost individual documents
(check the javadocs for Document and Field classes).

Are you aware of Nutch, though?  It sounds like you are not, and Nutch
is probably the best tool for the job (and it uses Lucene under the


--- Ivan Sekulovic <sekula@net.yu> wrote:

> Hello!
> I am currently choosing technology for web crawler and search engine 
> that will index between 1 and 10 million of documents (with storing 
> documents). For some parts of the project I'll most likely choose 
> existing software, for some I'll have to right new code, but at the
> end 
> it should be pure java solution.
> I am considering Lucene as solutions for text indexing and searching
> and 
> I have few questions about Lucene for which I was not able to find 
> answers in FAQs, Articles etc.
> Is Lucene suitable for ~10 million documents?
> Is it possible to have boosts factor per document ? The thing is that
> I 
> need to have something like sort order of documents in relevance, but
> relevance cannot been calculated only from that document, because
> there 
> are some external factors as well (e.g. Google PageRank algorithm). I
> think that I can calculate all this factors in one factor that can
> been 
> stored in index, but can I use it to boost relevance of some
> documents ?
> I guess it is possible, but would it require for some parts of Lucene
> to 
> be rewritten to enable this ? Or should I just fetch documents from 
> Lucene and then sort them outside?
> Best Regards,
> Sekula
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message