lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] [Commented] (LUCENE-4317) does not reuse its inlined Keyword-TokenStream
Date Tue, 21 Aug 2012 12:17:38 GMT


Uwe Schindler commented on LUCENE-4317:

bq. Mike: Or (crazy idea): maybe we could simply call on the analyzer (like we do for normal
tokenized fields), but then insist what was returned is in fact from KeywordAnalyzer? This
would force users of StringField to use PFAW w/ this field mapping to KeywordAnalyzer. It's
rather... anal though. And will be confusing to users who "forget" to use PFAW (but then this
is a service to them: it points out that at query-time their analysis is wrong). Advanced
users are free to use Field directly if somehow this checking becomes a problem ...

I disagree with that, this is too much magic and does not help with numerics again :-)

bq. Robert: i certainly feel like if we want to take String we could just apply StringReader
ourself by default (someone could override).

+1. Just another idea: The Analyzer can internally also reuse a ReuseableStringReader like
IndexWriter did in 3.x.

Please lets discuss all this in a separate issue and maybe for 5.0. Now it is too late for
that (in my opinion).
> does not reuse its inlined Keyword-TokenStream
> ---------------------------------------------------------
>                 Key: LUCENE-4317
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 4.0-BETA
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 5.0, 4.0
>         Attachments: LUCENE-4317.patch
> contains a inlined Keyword-TokenStream. Unfortunately this one is recreated
all the time, although one reuses the same Field instance. For NumericTokenStream
reuses it, but the Keyword one not.
> We should apply the same logic and lazy init the TokenStream with a setter for the String
value and reset(). This would be looking identical to SetNumeric(xx).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message