lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renaud Delbru <renaud.del...@deri.org>
Subject Re: Sorting posting lists before intersection
Date Mon, 13 Oct 2008 14:21:47 GMT
Hi Andrzej,

sorry for the late reply.

I have looked at the code. As far as I understand, you sort the posting 
lists based on the first doc skip. The first posting list will be the 
one who have the first biggest document skip.
Do the sparseness of posting lists is a good predictor for sampling and 
ordering posting lists ? Do you know evaluation of such technique ?

In order to implement sorting based on frequency, we need the document 
frequency of each term. This information should be propagated through 
the Scorer classes (from TermScorer to higher level class such as 
ConjunctiveScorer). This will require a call to 
IndexReader.docFreq(term) for each of the term queries. Is docFreq call 
mean another IO access ?

Thanks for the clarification,
Regards.
-- 
Renaud Delbru


Andrzej Bialecki wrote:
>
> Renaud Delbru wrote:
> > Hi all,
> >
> > I am wondering if Lucene implements the query optimisation that consists
> > of ordering the posting lists based on the term frequency before
> > intersection ?
> > If yes, could somebody point me to the java class / method that
> > implements such strategy ?
>
> Lucene trunk: ConjunctionScorer, lines 85-103 - pay attention to the
> comments there, it's not strictly a sort by frequency, rather by the
> sampled "sparseness".
>
> --
> Best regards,
> Andrzej Bialecki     <><
>   ___. ___ ___ ___ _ _   __________________________________
> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> ___|||__||  \|  ||  |  Embedded Unix, System Integration
> http://www.sigram.com  Contact: info at sigram dot com
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message