Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <472300E0.3070203@getopt.org>
Date: Sat, 27 Oct 2007 11:12:00 +0200
From: Andrzej Bialecki <ab@getopt.org>
User-Agent: Thunderbird 2.0.0.6 (Windows/20070728)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: Sorted Index
References: <13438928.post@talk.nabble.com>
 <c68e39170710262019m775cd99ax48cbcebf65e20e2@mail.gmail.com>
 <13439134.post@talk.nabble.com>
In-Reply-To: <13439134.post@talk.nabble.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

John Patterson wrote:
> 
> 
> Yonik Seeley wrote:
>> On 10/26/07, John Patterson <jdp2000@gmail.com> wrote:
>> Most things in an inverted index are sorted (terms, matching document
>> ids, term positions within a field, etc).  Can you be more specific
>> about what you are trying to accomplish?
>>
> 
> Sorry, I mean sorting the documents in an order other than the order they
> are added.  The my search could just return docs in index order.  For the
> most common sorting I could collect only the first x docs and then
> short-circuit the search like we previously discussed.

These questions already have an answer in Nutch (see the 
org.apache.nutch.indexer.IndexSorter, and 
org.apache.nutch.searcher.LuceneQueryOptimizer$LimitedCollector).

> 
> I was wondering if it is possible to apply a sort at merge time?

One method that I'm familiar with is the following: you can split the 
result set into several large-ish bins, and apply arbitrary sorting 
methods within each bin. Studies show that if you pick the right bin 
size, users will rarely look into the second and the following bins, so 
the task is reduced to the sorting of the first bin, e.g. 100 top 
scoring docs.

-- 
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org