lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ype Kingma <ykin...@xs4all.nl>
Subject RE: Sorting
Date Thu, 20 Jun 2002 19:45:48 GMT
Nader,

>I've been working with Lucene for the past three months and we're at t
>the final stages before launching with Lucene as our core search engine
>(the move was made from oracle internmedia) and sorting was an issue
>as we normally pumped out our results ordered by date. As it stands Lucene
>produces a resultant set ordered by Score, I believe that with some tweaking
>the same logic used to fetch score-orderd results could be applied to other
>fields.

But retrieving stored fields can be slow. I suppose you have the dates in
RAM?
The scores are computed from the index (see the low level search API),
avoiding retrieval of stored fields.

>I might be a little bit off here, but sorting the results in memory would be
>too intensive
>and would defeat the purpose as a 12 000 record resultant set would outweigh
>the speed provided
>by Lucene witch remains one of it's biggest assets, and trust me I tried
>Quick Sort ..
>Bi-Directional Quick Sort and a slew of other sorting algorithms, too slow
>if you want to scale
>well.

When you need to move disk heads during query evaluation, adding doc/date
pairs to a sorted map could use the CPU time in between.
Did you try inserting into a sorted map while lucene is computing
the next document score? Or would that give too many thread switches?

With a bit of luck you might have the sorted result ready very shortly
after the last document was processed.

Regards,
Ype

-- 

--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>


Mime
View raw message