lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raavan <valla...@gmail.com>
Subject Re: is OpenBitSet / SortedVIntList compressed bit map index?
Date Sat, 08 Jan 2011 18:24:36 GMT
Also, just for my understanding, is SortedVIntList able to perform some
operations such as AND/OR without decompression ?

Some of the algorithms mentioned below claim to do that. But I understand
that there are patent issues surrounding these algorithms.
http://en.wikipedia.org/wiki/Bitmap_index

-Raavan

On Sat, Jan 8, 2011 at 10:14 AM, Raavan <vallalku@gmail.com> wrote:

> Thanks Federico.
>
> >> my primary concern at the moment is serializing bitsets to recover
> searcher warmup time
>
> I am also considering doing the same to reduce warmup time during restarts.
>
> It seems one of the disadvantages of SortedVIntList is the performance
> skipTo() as per Paul Elschot since it does not support random access like
> OpenBitSet.
> https://issues.apache.org/jira/browse/LUCENE-1296
>
> Our primary concern is memory usage since we have hundreds of filters and
> large number of documents. So if the performance is decent, I am thinking of
> using SortedVIntList for all our sparse filters.
>
> -Raavan
>
>
> On Fri, Jan 7, 2011 at 3:11 PM, Federico Fissore <federico@fissore.org>wrote:
>
>> First Last, il 07/01/2011 20:55, ha scritto:
>>
>>  Hi,
>>>
>>> is OpenBitSet / SortedVIntList a compressed bit map index? Which one is
>>> better if memory usage is the primary concern ?
>>>
>>>
>> SortedVIntList is compressed, OpenBitSet is not
>>
>>
>>
>>  Our filters are sparse. So is SortedVIntList better in that case?
>>>
>>>
>> Yes
>>
>>
>>
>>  Are there any other compressed bitmap index implementations which offer
>>> bit
>>> map compression at a decent performance assuming filters are sparse?
>>>
>>>
>> I'm too looking for alternative implementations of compressed bitsets, so
>> I'm too really interested in everybody experience: my primary concern at the
>> moment is serializing bitsets to recover searcher warmup time
>>
>> I've tried some and roughly tested them: my conclusion was that we (lucene
>> users) already stand on the rolls royce of bitset implementations.
>>
>> federico
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message