lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <>
Subject [jira] Commented: (LUCENE-1796) Speed up repeated TokenStream init
Date Wed, 12 Aug 2009 11:28:14 GMT


Yonik Seeley commented on LUCENE-1796:

bq. But in principle we could also change the indexer to call clear before each incrementToken()
removing the need to do it in every Tokenizer.

Doron brought up a good reason for not doing that in LUCENE-1101.
A tokenizer (or other token producer) could produce multiple tokens before one made it to
the ultimate consumer (because of stop filters, etc).  So it looks like producers should do
the clear.

> Speed up repeated TokenStream init
> ----------------------------------
>                 Key: LUCENE-1796
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>         Attachments: after.png, afterAndLucene1796.png, before.png, LUCENE-1796.patch,
LUCENE-1796.patch, LUCENE-1796.patch, LUCENE-1796.patch, LUCENE-1796.patch
>  by caching isMethodOverridden results

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message