lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dingjin (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (LUCENE-3834) The tokenstream create by SmartChineseAnalyzer can't reset
Date Wed, 29 Feb 2012 07:40:03 GMT
The tokenstream create by SmartChineseAnalyzer can't reset
----------------------------------------------------------

                 Key: LUCENE-3834
                 URL: https://issues.apache.org/jira/browse/LUCENE-3834
             Project: Lucene - Java
          Issue Type: Bug
          Components: modules/analysis
    Affects Versions: 3.5
            Reporter: dingjin


That is because the field input in class SentenceTokenizer isn't reset after we call the method
reset().

They are two input field,one is from Tokenizer and another is from  TokenFilter,if we need
to reset a tokenstream created by SmartChineseAnalyzer, both  them need reset.This bug is
because of the author forget reset input field in class SentenceTokenizer .

class path : org.apache.lucene.analysis.cn.smart.SentenceTokenizer

oringal code

public final class SentenceTokenizer extends Tokenizer {
  ....
  @Override
  public void reset() throws IOException {
    super.reset();
    tokenStart = tokenEnd = 0;
  }

 ...
}

this method should changes as follow

 
  public void reset() throws IOException {
    super.reset();
    //should reset input
    input.reset();
    tokenStart = tokenEnd = 0;
  }





--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message