lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <>
Subject who clears attributes?
Date Mon, 10 Aug 2009 16:01:21 GMT
CharTokenizer.incrementToken() clears *all* attributes in the entire
tokenizer chain.
StandardTokenizer.incrementToken() clears only the term attribute.

So... which is right?  Seems like the tokenizer should be responsible?

On a performance related note, CharTokenizer.clearAttribtes() could be
more efficient - 2 new objects (the unmodifiable map and the iterator
object) are created for every incrementToken.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message