lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] [Commented] (LUCENE-4317) does not reuse its inlined Keyword-TokenStream
Date Tue, 21 Aug 2012 09:26:38 GMT


Uwe Schindler commented on LUCENE-4317:

bq. Do you mean also for NumericTokenStream? or just StringTokenStream?

Both, of course. A plain Analyzer will not help, as it expects a Reader :( And creating a
StringReader for every String is also a bad idea.

I was thinking about some Analyzer-Like CloseableThreadLocal, but the problem is, we can never
close it. IndexWriter previously managed to close the ThreadLocal (for its StringTokenStream-like
internal AS), but thats no longer possible here.
> does not reuse its inlined Keyword-TokenStream
> ---------------------------------------------------------
>                 Key: LUCENE-4317
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 4.0-BETA
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 5.0, 4.0
>         Attachments: LUCENE-4317.patch
> contains a inlined Keyword-TokenStream. Unfortunately this one is recreated
all the time, although one reuses the same Field instance. For NumericTokenStream
reuses it, but the Keyword one not.
> We should apply the same logic and lazy init the TokenStream with a setter for the String
value and reset(). This would be looking identical to SetNumeric(xx).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message