lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <yo...@lucidimagination.com>
Subject Re: Field Collapsing Performance
Date Wed, 29 Sep 2010 17:46:43 GMT
On Tue, Sep 28, 2010 at 8:14 PM, Li Li <fancyerii@gmail.com> wrote:
> I think current implmetation is slow. because it do collapse in all
> the hit docs. In my view, it will take more than 1s when using
> collapse and only 200ms-300ms when not in our environment. So we
> modify it as -- when user need top 100 docs, we collect top 200 docs
> and do collapse within these 200 docs.

Yep, like faceting, there's no one algorithm thats fast for all types
of distributions.
If you expect groups to be relatively unique, the most efficient way
is just for the
client to over-request a bit and do the collapse themselves.

We'll be adding more implementations as time goes on of course, but I think
tackling something first that the client *couldn't* easily do was a good choice.

-Yonik
http://lucenerevolution.org  Lucene/Solr Conference, Boston Oct 7-8

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message