lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raavan <valla...@gmail.com>
Subject Re: is OpenBitSet / SortedVIntList compressed bit map index?
Date Sat, 08 Jan 2011 18:14:23 GMT
Thanks Federico.

>> my primary concern at the moment is serializing bitsets to recover
searcher warmup time

I am also considering doing the same to reduce warmup time during restarts.

It seems one of the disadvantages of SortedVIntList is the performance
skipTo() as per Paul Elschot since it does not support random access like
OpenBitSet.
https://issues.apache.org/jira/browse/LUCENE-1296

Our primary concern is memory usage since we have hundreds of filters and
large number of documents. So if the performance is decent, I am thinking of
using SortedVIntList for all our sparse filters.

-Raavan

On Fri, Jan 7, 2011 at 3:11 PM, Federico Fissore <federico@fissore.org>wrote:

> First Last, il 07/01/2011 20:55, ha scritto:
>
>  Hi,
>>
>> is OpenBitSet / SortedVIntList a compressed bit map index? Which one is
>> better if memory usage is the primary concern ?
>>
>>
> SortedVIntList is compressed, OpenBitSet is not
>
>
>
>  Our filters are sparse. So is SortedVIntList better in that case?
>>
>>
> Yes
>
>
>
>  Are there any other compressed bitmap index implementations which offer
>> bit
>> map compression at a decent performance assuming filters are sparse?
>>
>>
> I'm too looking for alternative implementations of compressed bitsets, so
> I'm too really interested in everybody experience: my primary concern at the
> moment is serializing bitsets to recover searcher warmup time
>
> I've tried some and roughly tested them: my conclusion was that we (lucene
> users) already stand on the rolls royce of bitset implementations.
>
> federico
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message