lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <>
Subject analysis back compat break
Date Thu, 17 Sep 2009 23:57:44 GMT
Unfortunately it doesn't look like analysis is actually back compatible.

Consider the following token filter snippet that used to work fine:

  public Token next(Token target) throws IOException {;
    Token peek =;

What I'm seeing now is that the target and peek tokens are both the same.
This is because the "input" stream supports incrementToken and thus
incrementToken() is called first, changing "target".

Relevant code from TokenStream:
  public Token next() throws IOException {
    if (tokenWrapper == null)
      throw new UnsupportedOperationException("This TokenStream only
supports the new Attributes API.");

    if (supportedMethods.hasIncrementToken) {
      return incrementToken() ? ((Token) tokenWrapper.delegate.clone()) : null;


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message