Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of vallalku@gmail.com designates
 209.85.216.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=kHmYNH9rfyCwZpRd5lsVp22pY5g9KxyVfRBX9JToAETKAIBdSH/AO7Qk9DwlbkKf2T
         i0HOvv5hrKfGDx16ekIJv6gkEBEB3vnnhVA/l7KF1DAyQt1rLUHk7O7rnC2xxTmFOKDu
         djIsETEfT/AqN4BwoLR3cDBK1UhyztwlBVkwQ=
MIME-Version: 1.0
In-Reply-To: <AANLkTikvi9uS87WZRxnXEGBwjaigkFq4MkHqJK0MBofO@mail.gmail.com>
References: <AANLkTimnD9Ma+3=SzWbqxt4ct9urxiG819yCTOJhO0ob@mail.gmail.com>
	<4D279DA9.70506@fissore.org>
	<AANLkTikvi9uS87WZRxnXEGBwjaigkFq4MkHqJK0MBofO@mail.gmail.com>
Date: Sat, 8 Jan 2011 10:24:36 -0800
Message-ID: <AANLkTi=hMBQKmD-nuK8Gb=Uaa_xm4F19CQAWjMVQi4z7@mail.gmail.com>
Subject: Re: is OpenBitSet / SortedVIntList compressed bit map index?
From: Raavan <vallalku@gmail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=001485f92356e11df6049959d80a

--001485f92356e11df6049959d80a
Content-Type: text/plain; charset=ISO-8859-1

Also, just for my understanding, is SortedVIntList able to perform some
operations such as AND/OR without decompression ?

Some of the algorithms mentioned below claim to do that. But I understand
that there are patent issues surrounding these algorithms.
http://en.wikipedia.org/wiki/Bitmap_index

-Raavan

On Sat, Jan 8, 2011 at 10:14 AM, Raavan <vallalku@gmail.com> wrote:

> Thanks Federico.
>
> >> my primary concern at the moment is serializing bitsets to recover
> searcher warmup time
>
> I am also considering doing the same to reduce warmup time during restarts.
>
> It seems one of the disadvantages of SortedVIntList is the performance
> skipTo() as per Paul Elschot since it does not support random access like
> OpenBitSet.
> https://issues.apache.org/jira/browse/LUCENE-1296
>
> Our primary concern is memory usage since we have hundreds of filters and
> large number of documents. So if the performance is decent, I am thinking of
> using SortedVIntList for all our sparse filters.
>
> -Raavan
>
>
> On Fri, Jan 7, 2011 at 3:11 PM, Federico Fissore <federico@fissore.org>wrote:
>
>> First Last, il 07/01/2011 20:55, ha scritto:
>>
>>  Hi,
>>>
>>> is OpenBitSet / SortedVIntList a compressed bit map index? Which one is
>>> better if memory usage is the primary concern ?
>>>
>>>
>> SortedVIntList is compressed, OpenBitSet is not
>>
>>
>>
>>  Our filters are sparse. So is SortedVIntList better in that case?
>>>
>>>
>> Yes
>>
>>
>>
>>  Are there any other compressed bitmap index implementations which offer
>>> bit
>>> map compression at a decent performance assuming filters are sparse?
>>>
>>>
>> I'm too looking for alternative implementations of compressed bitsets, so
>> I'm too really interested in everybody experience: my primary concern at the
>> moment is serializing bitsets to recover searcher warmup time
>>
>> I've tried some and roughly tested them: my conclusion was that we (lucene
>> users) already stand on the rolls royce of bitset implementations.
>>
>> federico
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>

--001485f92356e11df6049959d80a--