lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <>
Subject Re: Re Lucene analyzers
Date Wed, 10 Dec 2008 21:52:08 GMT

: public final TokenStream tokenStream(String fieldName, Reader reader)
: Usually does  a bunch of new filters, from what I seen in most of these
: filters none of the use class member variables.  Has anybody tried making
: them static to avoid the creation of new objects.

that wouldn't really work -- partly because then your Analyzer wouldn't be 
thread safe (so you couldn't index more then one doc in parallel) and 
partly because all those Tokenizers and TokenFilter's take the TokenStream 
as a constructor arg.

Note that anytime an Analyzer needs to do some initialization that's 
common for all TokenStreams (like building up a Stop Words Set) it does it 
once in it's own constructor, so the object construction cost in the 
tokenStream method calls is relatively cheap.

Of course: much of this is changing in 2.9/3.0 .. in ways i still don't 
fully understand, so stay tuned.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message