lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trejkaz <trej...@trypticon.org>
Subject Re: Efficient way to define large Boolean Occur.FILTER clause in Lucene 6
Date Wed, 27 Jun 2018 02:51:08 GMT
On Tue, Jun 26, 2018 at 7:02 PM, Hasenberger, Josef
<Josef.Hasenberger@zetcom.com> wrote:
> However, I have a feeling that the conversion from Long values to Terms is
> rather inefficient for large collections and also uses a lot of memory.
> To ease conversion overhead somewhat, I created a class that converts a
> Long value directly to BytesRef instance (in order to avoid conversion to
> UTF16 and then UTF8 again) and pass that instance to the Term constructor.

First thought is, why are you using TermsQuery if they're in DocValues?
Is DocValuesTermsQuery any better? It does depend on how many terms
you're searching for.

Second thought is that there is also DocValuesNumbersQuery, which
avoids having to convert all the values.

> I just wonder if there is a better method for passing large amount of filter criteria
> to a BooleanQuery Occur.FILTER clause, that avoids excessive object creation.

If you can get your long values into something which implements Bits,
you could make a query using RandomAccessWeight to directly point at
the existing set you already have in memory.

TX

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message