From lucene-dev-return-3101-qmlist-jakarta-archive-lucene-dev=nagoya.apache.org@jakarta.apache.org Tue Feb 11 14:38:31 2003 Return-Path: Delivered-To: apmail-jakarta-lucene-dev-archive@apache.org Received: (qmail 7026 invoked from network); 11 Feb 2003 14:38:30 -0000 Received: from exchange.sun.com (192.18.33.10) by daedalus.apache.org with SMTP; 11 Feb 2003 14:38:30 -0000 Received: (qmail 2132 invoked by uid 97); 11 Feb 2003 14:40:01 -0000 Delivered-To: qmlist-jakarta-archive-lucene-dev@nagoya.betaversion.org Received: (qmail 2125 invoked from network); 11 Feb 2003 14:40:01 -0000 Received: from daedalus.apache.org (HELO apache.org) (208.185.179.12) by nagoya.betaversion.org with SMTP; 11 Feb 2003 14:40:01 -0000 Received: (qmail 6715 invoked by uid 500); 11 Feb 2003 14:38:26 -0000 Mailing-List: contact lucene-dev-help@jakarta.apache.org; run by ezmlm Precedence: bulk List-Unsubscribe: List-Subscribe: List-Help: List-Post: List-Id: "Lucene Developers List" Reply-To: "Lucene Developers List" Delivered-To: mailing list lucene-dev@jakarta.apache.org Received: (qmail 6636 invoked from network); 11 Feb 2003 14:38:24 -0000 Received: from mxzilla2.xs4all.nl (194.109.6.50) by daedalus.apache.org with SMTP; 11 Feb 2003 14:38:24 -0000 Received: from webmail5.xs4all.nl (webmail5.xs4all.nl [194.109.127.34]) by mxzilla2.xs4all.nl (8.12.3/8.12.3) with ESMTP id h1BEcPJ4079010 for ; Tue, 11 Feb 2003 15:38:25 +0100 (CET) Received: (from nobody@localhost) by webmail5.xs4all.nl (8.11.6/8.11.1) id h1BEcPj09788; Tue, 11 Feb 2003 15:38:25 +0100 (CET) (envelope-from ykingma@xs4all.nl) X-Authentication-Warning: webmail5.xs4all.nl: nobody set sender to ykingma@xs4all.nl using -f Received: from 212.206.88.5 (SquirrelMail authenticated user ykingma) by webmail.xs4all.nl with HTTP; Tue, 11 Feb 2003 15:38:25 +0100 (CET) Message-ID: <5542.212.206.88.5.1044974305.squirrel@webmail.xs4all.nl> In-Reply-To: <3E4401CE.9040808@lucene.com> References: <200302032250.h13MnxJH079371@smtpzilla5.xs4all.nl> <3E4401CE.9040808@lucene.com> Date: Tue, 11 Feb 2003 15:38:25 +0100 (CET) Subject: Re: MultiSearcher discards interim results From: ykingma@xs4all.nl To: "Lucene Developers List" X-Mailer: SquirrelMail (version 1.3.2 [CVS-DEVEL]) MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Doug, Sorry for the delay. I'm having a hardware problem on my home machine and I'm using webmail now. It might take some time before I can continue. > I'm confused. The contract of this method is to return the top-scoring > nDocs. For a multi-searcher it must compute the top-scoring nDocs from > each sub-searcher, then find the top-scoring nDocs among these. For the first sub-searcher, yes. For later sub-searchers it is only necessary to keep the documents that score not smaller than the current minimum score. Worst case: consider what happens when later subsearchers only find scores smaller than the minimum score kept by the first sub-searcher. In that case the current code builds up a full ndocs size priority queue for each later subsearcher, and all these results are going to be discarded. The patch intends to avoid the housekeeping of the nDocs size priority queues for the later sub-searchers by using a single priority queue for all sub-searchers. The story behind this is that at some point I actually had a very bad implementation of a multi searcher and retriever using a TopDocs result from each subsearcher. The net effect was that not only where all results kept, but also all the stored results had to be retrieved, before discarding most of them. Needless to say that I switched to a home grown HitCollector very soon... As the current MultiSearcher also provides consistent scoring between databases, I'm going to use it asap. Are people actually using the nightly builds? I'd also like to give the scoring explanation facilities a try. Kind regards, Ype Kingma --------------------------------------------------------------------- To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org For additional commands, e-mail: lucene-dev-help@jakarta.apache.org