lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init
Date Tue, 11 Aug 2009 16:53:14 GMT


Uwe Schindler commented on LUCENE-1796:

bq. I don't know if all of the Tokenizers in lucene were ever changed, but in any case it
looks like at least some of these bugs were introduced with the switch to the attribute API
- for example StandardTokenizer did clear it's reusableToken... and now it doesn't.

No one is calling clearAttributes() in trunk code, only some of them clear attributes before
filling data in.

OK, I open another issue later and change all Tokenizers in core and contrib to call clearAttributes()
as first call inside incrementToken()?

But in principle we could also change the indexer to call clear before each incrementToken()
removing the need to do it in every Tokenizer.

> Speed up repeated TokenStream init
> ----------------------------------
>                 Key: LUCENE-1796
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>         Attachments: after.png, afterAndLucene1796.png, before.png, LUCENE-1796.patch,
LUCENE-1796.patch, LUCENE-1796.patch, LUCENE-1796.patch, LUCENE-1796.patch
>  by caching isMethodOverridden results

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message