Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: local policy includes SPF record at
 spf.trusted-forwarder.org)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.co.uk;
  h=Message-ID:Received:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type;
  b=VkdGebL0z3344nPmJZ/rZmvwKhbpocFi2xDxrzBj3VdiCB28z9uKu2CsRv00E5lQPQMtHEQMdQYgOKmtM3ZG1EQFOz0RLRBbEeM9ioFuYgDN2zvvf7layxuIupqJpgaIr8Sn4JivVRNRjZmbYOGTdecRmbV54EfZhfSF0a4mY7I=
  ;
Message-ID: <20060906061031.36941.qmail@web25912.mail.ukl.yahoo.com>
Date: Wed, 6 Sep 2006 06:10:31 +0000 (GMT)
From: eks dev <eksdev@yahoo.co.uk>
Reply-To: eks dev <eksdev@yahoo.co.uk>
Subject: Re: [jira] Updated: (LUCENE-584) Decouple Filter from BitSet
To: java-dev@lucene.apache.org
In-Reply-To: <c68e39170609051415r6fd4106au1f4759e4d3b65040@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii


Wierd indeed! I tend to beleive I made something stupid in Matcher test, but it looks too simple (OpenBitsMatcher and BitsMatcher are identical?!). I even itterated the same test many times, and there is no big difference   in results, speed curve is only not so jittery like on single itteration...

humpf,  wild guess out of my league: maybe  some instruction cache  in cpu in slightly more complex execution path breaks your optimization.?


With your test it is rather clear who wins:

-Xmx128m -server -Xbatch 8000000 5 7500000 nextSetBit 10 bit
ret=-1704152202
TIME=10996

-Xmx128m -server -Xbatch 8000000 5 7500000 nextSetBit 10 open
ret=-1704152202
TIME=8092

Jvm 32 bit "build 1.6.0-rc-b94"
 CPU Intel Pentium M 1.60GHz notebook
 

----- Original Message ----
From: Yonik Seeley <yonik@apache.org>
To: java-dev@lucene.apache.org
Sent: Tuesday, 5 September, 2006 11:15:42 PM
Subject: Re: [jira] Updated: (LUCENE-584) Decouple Filter from BitSet

On 9/4/06, Eks Dev (JIRA) <jira@apache.org> wrote:

> Here are some Matcher implementations,
>
> - OpenBitsMatcher- the same as the code Paul wrote for BitsMatcher, with replaced OpenBitSet instead
>
> -DenseOpenBitsMatcher  - Using solr BitSetIterator (for skipTo() to work, one method in BitSetIterator should become public)

Keep in mind that BitSetIterator is fast for iteration over all it's bits.
If it's used as a filter (with skipping), I would expect it to be slower.

> Also attached one simple  test (just basic fuctionality) that also contains one dummy relative performance  test
>
> Perf. test simply iterates over different Matcher implementations  and measures ellapsed time (not including Matcher creation, pure forward scan to the end) for different set bit densities.
>
> imho, this code is not sufficiantly tested nor commented, needs an hour or two.
>
> As expected, Yonik made this BitSetIterator really fast. What was surprise for me was OpenBitSet nextSetBit() comparing bad to the BitSet  (or I made some dummy mistake somewhere?)

That's weird... what CPU and in what mode (32 or 64 bit)?  What JVM params?
Do you get the same results from org.apache.solr.util.BitSetPerf for nextSetBit?

I did write it on a P4, and ntz (number of trailing zeros) is
currently optimized for a 32 bit CPU where a 64 bit shift may be
slightly more expensive.

-Yonik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org