lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <>
Subject RE: Clarification on TokenStream.close() needed
Date Tue, 20 Oct 2009 19:48:33 GMT
TokenStream.close() is called (and was everytime called before, too), when
the tokenization is done to close the Reader. The call to reset(Reader) is
the same like creating a new instance (only that the cost of creating a new
instance is not needed).

The change in Solr 1.4 is that now TokenStreams are reused, if the Analyzer
supports it. You should release all resources in close and recreate on
reset(). If this is too costly, they should not be released in close() but
then there is no way to release them (only in finalize() which is called by


Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen

> -----Original Message-----
> From: Teruhiko Kurosaka []
> Sent: Tuesday, October 20, 2009 9:27 PM
> To:
> Subject: Clarification on TokenStream.close() needed
> Hi,
> My Tokenizer started showing an error when I switched
> to Solr 1.4 dev version.  I am not too confident but
> it seems that Solr 1.4 calls close() on my Tokenizer
> before calling reset(Reader) in order to reuse
> the Tokenizer.  That is, close() is called more than
> once.
> The API doc of close() reads:
> Releases resources associated with this stream.
> So I thought close() shold be called only once,
> and the Tokenizer objects cannot be reused after
> close() is called.  Is my interpretation correct?
> If my interpretation is wrong and it is legal to
> call close() more than once, where is the best place
> to free per-instance resources?
> T. "Kuro" Kurosaka
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message