lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Is creating an analyzer expensive?
Date Fri, 17 Aug 2012 21:39:16 GMT
You have to use the TokenStream retrieved by Analyzer in the specified
order, otherwise it will not work correctly and will behave as described by
you:

reset()
while (incrementToken())
end()
close()

You have to call reset() also when using for first time! That's specified in
the specs. If you do this, it will work as expected.
See:
https://builds.apache.org/job/Lucene-Artifacts-4.x/javadoc/core/org/apache/l
ucene/analysis/TokenStream.html
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: rrs [mailto:rrs@rand.org]
> Sent: Friday, August 17, 2012 6:38 PM
> To: java-user@lucene.apache.org
> Subject: Re: Is creating an analyzer expensive?
> 
> Hi Simon,
> 
> I'm trying to reuse a custom analyzer and it's not working unless I
manually call
> reset() on the TokenStream. Basically the analyzer will work on the first
string,
> but complete fail on any string after that. The weird part is that this is
only
> necessary when using the SynonymFilter.
> 
> I wrote a short piece of code that shows what's going on. Let me know if
you
> want me to post the code. But it's more likely that I don't understand
what I'm
> doing - I am new to Lucene.
> 
> Thank you,
> 
> -ricardo
> 
> 
> 
> --
> View this message in context:
http://lucene.472066.n3.nabble.com/Is-creating-
> an-analyzer-expensive-tp3994731p4001878.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message