Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (hermes.apache.org: local policy)
Message-ID: <46038975.8020204@teamware.com>
Date: Fri, 23 Mar 2007 19:01:57 +1100
From: Antony Bowesman <adb@teamware.com>
Organization: Teamware Group
User-Agent: Thunderbird 1.5.0.10 (Windows/20070221)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: Combining score from two or more hits
References: <460219CC.6020808@teamware.com>
 <359a92830703220607u5cf8278cr857a050f27504414@mail.gmail.com>
 <4602DE2E.2020306@teamware.com>
 <Pine.LNX.4.58.0703222206390.32431@hal.rescomp.berkeley.edu>
In-Reply-To: <Pine.LNX.4.58.0703222206390.32431@hal.rescomp.berkeley.edu>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Chris Hostetter wrote:
> 
> if you are using a HitCollector, there any re-evaluation is going to
> happen in your code using whatever mechanism you want -- once your collect
> method is called on a docid, Lucene is done with that docid and no longer
> cares about it ... it's only whatever storage you may be maintaining of
> high scoring docs thta needs to know that you've decided the score has
> changed.
> 
> your big problem is going to be that you basically need to maintain a list
> of *every* doc collected, if you don't know what the score of any of them
> are until you've processed all the rest ... since docs are collected in
> increasing order of docid, you might be able to make some optimizations
> based on how big of a gap you've got between the doc you are currently
> collecting and the last doc you've collected if you know that you're
> always going to add docs that "relate" to eachother in sequential bundles
> -- but this would be some very custom code depending on your use case.

I only ever need to return a couple of ID fields per doc hit, so I load them 
with FieldCache when I start a new searcher.  These IDs refer to unique objects 
elsewhere, but there can be one or more instances of the same Id in the index 
due to the way I've structured Documents.  A Document = an attachment in the 
other system attached to the other system's object which can have 1...n 
attachments.  My problem is I need to return only unique external Ids with some 
kind of combined score up to the requested maxHits from the client.

Getting the unique Ids is no problem, but as you say I either have to store all 
hits and then sort them by score at the end once I know all unique docs, or do 
some clever stuff with some type of PriorityQueue that allows me to re-jig 
scores that already exist in the sorted queue.

One idea your comments raise is the relationship of docids to the group of 
Documents added for the higher level object.  All the Documents for the external 
object are added with a single writer at index time.  Assuming that the 
Documents for a single external Id will either all exist or none, then will the 
doc ids always be sequential for ever for that external Id or will they 
'reorganise' themselves?

Thanks
Antony


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org