lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <>
Subject RE: who clears attributes?
Date Mon, 10 Aug 2009 17:23:03 GMT
> On Mon, Aug 10, 2009 at 12:44 PM, Uwe Schindler<> wrote:
> >the CharTokenizer should only clear the TermAttribute, as it is only
> using this attribute.

I changed this in the latest patch for

> It's certainly not clear to me - is there an established convention?
> Either Tokenizer clears all attributes, or each tokenizer clears those
> attributes it cares about.  But in the latter case, wouldn't that
> potentially cause multiple TokenFilters to clear the same attribute?

Clearing attributes in TokenFilters is not the best. The problem is, that
calling clear() on an AttributeImpl may not only clear the directly
referenced values, the multi-attribute implementations like
Token/TokenWrapper currently used, always clear all 6 standard attributes.
Because of this, I would only clear attributes in TokenStream/Tokenizer, but
then per default for all Tokenizers. Maybe we should implement this. The
problem with that is still the iterator creation, but I have no better
solution as Maps only work with iterators for enumerating values... :(


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message