lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Keegan" <peterlkee...@gmail.com>
Subject Re: Aggregating category hits
Date Wed, 14 Jun 2006 15:39:06 GMT
The performance results in my previous posting were based on an
implementation that performs 2 searches, one for getting 'Hits' and another
for getting the BitSet. I reimplemented this in one search using the code in
'SolrIndexSearcher.getDocListAndSetNC' and I'm now getting throughput of
350-375 qps.

This is great stuff Solr guys! I'd love to see the DocSet and DocList
features added to Lucene's IndexSearcher.

Peter

On 6/12/06, Peter Keegan <peterlkeegan@gmail.com> wrote:
>
> I'm seeing query throughput of approx. 290 qps with OpenBitSet vs. 270
> with BitSet. I had to reduce the max. HashDocSet size to 2K - 3K (from
> 10K-20K) to get optimal tradeoff.
>
> no. docs in index: 730,000
> average no. results returned: 40
> average response time: 50 msec (15-20 for counting facets)
> no. facets: 100 on every query
>
> I'm not using the Solr server as we have already developed an
> infrastructure.
>
> Peter
>
>
>
> On 6/10/06, Yonik Seeley <yseeley@gmail.com> wrote:
> >
> > On 6/9/06, Peter Keegan <peterlkeegan@gmail.com> wrote:
> > > However, my throughput testing shows that the Solr method is at least
> > 50%
> > > faster than mine. I'm seeing a big win with the use of the HashDocSet
> > for
> > > lower hit counts. On my 64-bit platform, a MAX_SIZE value of 10K-20K
> > seems
> > > to provide optimal performance.
> >
> > Interesting... how many documents are in your collection?
> > It would prob be nice to make the HashDocSet cutt-off dynamic rather
> > than fixed.
> > Are you using Solr, or just some of it's code?
> >
> > >  I'm looking forward to trying this with
> > > OpenBitSet.
> >
> > I checked in the OpenBitSet changes today.  I imagine this will lower
> > the optimal max HashDocSet size for performance a little.  You might
> > not see much performance improvement if most of the intersections
> > involved a HashDocSet... the OpenBitSet improvements only kick in with
> > bitset<->bitset intersection counts.
> >
> > -Yonik
> > http://incubator.apache.org/solr Solr, the open-source Lucene search
> > server
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message