lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Muir <>
Subject Re: reset versus setReader on TokenStream
Date Wed, 29 Aug 2012 20:00:10 GMT
On Wed, Aug 29, 2012 at 3:54 PM, Benson Margulies <> wrote:
>  Some interlinear commentary on the doc.
> * Resets this stream to the beginning.
> To me this implies a rewind.  As previously noted, I don't see how this
> works for the existing implementations.

its not a rewind. the javadocs here are not good. we need to fix them
to be clear :)

>    * As all TokenStreams must be reusable,
>    * any implementations which have state that needs to be reset between
> usages
>    * of the TokenStream, must implement this method. Note that if your
> TokenStream
>    * caches tokens and feeds them back again after a reset,
> What's the alternative? What happens with all the existing Tokenizers that
> have no special implementation of #reset()?

perhaps these Tokenizers have no state to reset()? lots of tokenstream
classes are stateless.
if you are stateless, then you dont need to implement this method. You
get the default implementation: e.g. TokenFilter's just passes it down
the chain (input.reset()), and i think Tokenizer/TokenStream are


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message