See Tokenizer.java for the state machine logic. In general you should
not have to do anything if the tokenizer is well-behaved (e.g. close
calls super.close() and so on).
On Tue, Jan 7, 2014 at 2:50 PM, Benson Margulies <bimargulies@gmail.com> wrote:
> In 4.6.0, org.apache.lucene.analysis.BaseTokenStreamTestCase#checkResetException
>
> fails if incrementToken fails to throw if there's a missing reset.
>
> How am I supposed to organize this in a Tokenizer? A quick look at
> CharTokenizer did not reveal any code for the purpose.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
|