lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eks dev <>
Subject Re: [jira] Updated: (LUCENE-584) Decouple Filter from BitSet
Date Wed, 06 Sep 2006 06:10:31 GMT

Wierd indeed! I tend to beleive I made something stupid in Matcher test, but it looks too
simple (OpenBitsMatcher and BitsMatcher are identical?!). I even itterated the same test many
times, and there is no big difference   in results, speed curve is only not so jittery like
on single itteration...

humpf,  wild guess out of my league: maybe  some instruction cache  in cpu in slightly more
complex execution path breaks your optimization.?

With your test it is rather clear who wins:

-Xmx128m -server -Xbatch 8000000 5 7500000 nextSetBit 10 bit

-Xmx128m -server -Xbatch 8000000 5 7500000 nextSetBit 10 open

Jvm 32 bit "build 1.6.0-rc-b94"
 CPU Intel Pentium M 1.60GHz notebook

----- Original Message ----
From: Yonik Seeley <>
Sent: Tuesday, 5 September, 2006 11:15:42 PM
Subject: Re: [jira] Updated: (LUCENE-584) Decouple Filter from BitSet

On 9/4/06, Eks Dev (JIRA) <> wrote:

> Here are some Matcher implementations,
> - OpenBitsMatcher- the same as the code Paul wrote for BitsMatcher, with replaced OpenBitSet
> -DenseOpenBitsMatcher  - Using solr BitSetIterator (for skipTo() to work, one method
in BitSetIterator should become public)

Keep in mind that BitSetIterator is fast for iteration over all it's bits.
If it's used as a filter (with skipping), I would expect it to be slower.

> Also attached one simple  test (just basic fuctionality) that also contains one dummy
relative performance  test
> Perf. test simply iterates over different Matcher implementations  and measures ellapsed
time (not including Matcher creation, pure forward scan to the end) for different set bit
> imho, this code is not sufficiantly tested nor commented, needs an hour or two.
> As expected, Yonik made this BitSetIterator really fast. What was surprise for me was
OpenBitSet nextSetBit() comparing bad to the BitSet  (or I made some dummy mistake somewhere?)

That's weird... what CPU and in what mode (32 or 64 bit)?  What JVM params?
Do you get the same results from org.apache.solr.util.BitSetPerf for nextSetBit?

I did write it on a P4, and ntz (number of trailing zeros) is
currently optimized for a 32 bit CPU where a 64 bit shift may be
slightly more expensive.


To unsubscribe, e-mail:
For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message