lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <yo...@apache.org>
Subject Re: Poor performance "race condition" in FieldSortedHitQueue
Date Wed, 09 Aug 2006 02:47:35 GMT
The nature of the field cache itself means that the first sort on a
particular field can take a long, long time.  Synchronization won't
really help that much.

There are two ways around this...
1) incrementally generate the field cache (hard... not currently
supported by Lucene)
2) warm searchers in the background before exposing them to live
queries (the approach Solr takes).

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server


On 8/8/06, hutchiko@gmail.com <hutchiko@gmail.com> wrote:
> Hey all, just want to run an issue that I've recently identified while
> looking at some performance issues we are having with our larger
> indexes past you all.
>
> Basically what we are seeing is that when there are a number of
> concurrent searches being executed over a new IndexSearcher, the quite
> expensive ScoreDocComparator generation that is done in the
> FieldSortedHitQueue#getCachedComparator method ends up executing
> multiple times rather the ideal case of once. This issue does not
> effect the correctness of the searches only performance.
>
> For my relatively weak understanding of the code the core of this
> issue appears to lie with the FieldCacheImpl#getStringIndex method
> which allows multiple concurrent requests to each generate their own
> StringIndex rather than allowing the first request to do the
> generation and then blocking subsequent requests until the first
> request has finished.
>
> Is this a know problem? Should I raise this as an issue or is this
> "expected" behaviour. A solution would naturally require more
> synchronization than is currently used but nothing particularly
> complex.
>
> Thanks,
>
> Oliver

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message