lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trejkaz <>
Subject Re: Searching for Empty Field
Date Fri, 15 Jul 2011 07:00:02 GMT
On Fri, Jul 15, 2011 at 4:45 PM, Uwe Schindler <> wrote:
> Hi,
>> The crappy thing is that to actually detect if there are any tokens in the field
>> you need to make a TokenStream which can be used to read the first token
>> and then rewind again.  I'm not sure if there is such a thing in Lucene at the
>> moment.  We had to write it ourselves but we were on a considerably older
>> version at the time.
> CachingTokenFilter plugged over any other TokenStream.

Ah, quite right.  If you can afford the memory it will eat (or if your
documents are all relatively small), CachingTokenFilter will work.  I
think in our case it caused OOME for larger character streams, which
is why we ended up falling back to one which only cached the first


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message