lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Confusion with Analyzer.tokenStream() re-use in 4.1
Date Thu, 28 Feb 2013 07:40:30 GMT
The problem is how you use the Document/Field/Analyzer in your test code (see my mail that
explains it). The second problem is that you use new Field(...,TokenStream), which instantiates
the TokenStream at the time of calling, so it is "in use", which violates the general call-order
of Analyzers: Anylzers are fetched by IndexWriter consumed and given back to Analyzer. Here
you get the analyzer to early, confusing the other TokenStreams.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Konstantyn Smirnov [mailto:injecteer@yahoo.com]
> Sent: Thursday, February 28, 2013 12:18 AM
> To: java-user@lucene.apache.org
> Subject: RE: Confusion with Analyzer.tokenStream() re-use in 4.1
> 
> Thanks for the answer Uwe!
> 
> so the behavior has changed since the 3.6, hasn't it?
> 
> Now I need to instantiate the analyzer each time I feed the field with the
> tokenStream, or it happens behind the scenes if I use new (String name,
> String value, Field.Store store).
> 
> Another question then... Now I try my best to reuse the Document and Field
> instances when indexing more than 1 document. Is the instantiation of an
> analyzer heavy enough and shouldn't it be also re-used?
> 
> 
> 
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Confusion-with-Analyzer-
> tokenStream-re-use-in-4-1-tp4043427p4043508.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message