lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Keegan" <peterlkee...@gmail.com>
Subject Re: Aggregating category hits
Date Mon, 12 Jun 2006 16:43:49 GMT
I'm seeing query throughput of approx. 290 qps with OpenBitSet vs. 270 with
BitSet. I had to reduce the max. HashDocSet size to 2K - 3K (from 10K-20K)
to get optimal tradeoff.

no. docs in index: 730,000
average no. results returned: 40
average response time: 50 msec (15-20 for counting facets)
no. facets: 100 on every query

I'm not using the Solr server as we have already developed an
infrastructure.

Peter


On 6/10/06, Yonik Seeley <yseeley@gmail.com> wrote:
>
> On 6/9/06, Peter Keegan <peterlkeegan@gmail.com> wrote:
> > However, my throughput testing shows that the Solr method is at least
> 50%
> > faster than mine. I'm seeing a big win with the use of the HashDocSet
> for
> > lower hit counts. On my 64-bit platform, a MAX_SIZE value of 10K-20K
> seems
> > to provide optimal performance.
>
> Interesting... how many documents are in your collection?
> It would prob be nice to make the HashDocSet cutt-off dynamic rather than
> fixed.
> Are you using Solr, or just some of it's code?
>
> >  I'm looking forward to trying this with
> > OpenBitSet.
>
> I checked in the OpenBitSet changes today.  I imagine this will lower
> the optimal max HashDocSet size for performance a little.  You might
> not see much performance improvement if most of the intersections
> involved a HashDocSet... the OpenBitSet improvements only kick in with
> bitset<->bitset intersection counts.
>
> -Yonik
> http://incubator.apache.org/solr Solr, the open-source Lucene search
> server
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message