Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of teddyyyy123@gmail.com
 designates 209.85.215.48 as permitted sender)
MIME-Version: 1.0
From: Yang <teddyyyy123@gmail.com>
Date: Wed, 25 Apr 2012 14:13:06 -0700
Message-ID: 
 <CAAnh3_8p1hvLionR+-PAkWa0KavggFgGK=DREJZwwDnNZC7U-g@mail.gmail.com>
Subject: lucene algorithm ?
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=bcaec54858109d74a504be875763

--bcaec54858109d74a504be875763
Content-Type: text/plain; charset=ISO-8859-1

I read the paper by Doug "Space optimizations for total ranking",

since it was written a long time ago, I wonder what algorithms lucene uses
(regarding postings list traversal and score calculation, ranking)


particularly the total ranking algorithm described there needs to traverse
down the entire postings list for all the query terms,
so in case of very common query terms like "yellow dog", either of the 2
terms may have a very very long postings list in case of web search,
are they all really traversed in current lucene/Solr ? or  any heuristics
to truncate the list are actually employed?

in the case of returning top-k results, I can understand that partitioning
the postings list into multiple machines, and then combining the  top-k
from each would work,
but if we are required to return "the 100th result page", i.e. results
ranked from 990--1000th, then each partition would still have to find out
the top 1000, so
partitioning would not help much.


overall, is there any up-to-date detailed docs on the internal algorithms
of lucene?

Thanks a lot
Yang

--bcaec54858109d74a504be875763--