lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shay Banon <>
Subject RE: CachingTokenFilter#fillCache problem
Date Sat, 05 Sep 2009 11:22:22 GMT

Ohh, now I see. I missed the fact that everybody across the chain share the
same attributes. Which explain my bug (I forgot to call super at my
tokenizer, thus the attributes were not shared).


Uwe Schindler wrote:
> Hi Shay,
>>    I was going over and playing with Lucene 2.9 and came around something
>> strange that I think might be a bug. In CachingTokenFilter#fillCache it
>> seems like captureState is not called on input and instead is called on
>> CachingTokenFilter.
> This is not a bug and correct behaviour. In general it doesn't matter
> where
> you call captureState(), because all streams and filters in one tokenizer
> chain use exactly the same attribute instances. Calling
> input.incrementToken() moves forward to next token and after that the
> attributes in every filter on the chain contain the same term - just
> because
> they are the same instances.
>> Am I missing something? I got to it since QueryParser
>> uses the CachingTokenFilter.
> You miss the fact of the shared attributes :-)
> Thanks for looking into this and reviewing the code!
> Uwe
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View this message in context:
Sent from the Lucene - Java Developer mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message